Apparatus, method and article for evaluating a stack of objects in an image

ABSTRACT

An object evaluation system may determine a value for a stack of objects that appear in a pixelated color image. To determine the value of the stack of objects, the evaluation system preprocess at least a portion of the pixelated color image to produce a set of two color contour data, processes the two color contour data to identify a location of a top and a bottom of the stack (if any), and locates, for each of the objects in the stack, a respective set of color pixels from the pixelated color image corresponding to each object based on the identified locations of the top and bottom of the stack. Each of the objects in the stack are then classified into a color classification based on the object&#39;s respective set of color pixels, and the value of the object is determined based on a known correspondence between the color classification and a value. The cumulative value of the stack is determined by summing the determined values for each of the objects in the stack.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit under 35 U.S.C. §119(e) of U.S.Provisional patent application Ser. No. ______ filed Aug. 26, 2009(previously U.S. Nonprovisional patent application Ser. No. 12/548,289and converted by Petition To Convert granted by Decision withnotification date of Apr. 29, 2010); where this provisional applicationis incorporated herein by reference in its entirety.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

BACKGROUND

1. Field

This disclosure generally relates to image processing, and particularlyto computationally processing images of a stack of objects.

2. Description of the Related Art

Many businesses need to verify, inventory account or otherwise evaluatefor merchandise, currency, tokens or other objects in the course of theoperations of the business.

For example, a business may need to verify, inventory or account formerchandise in a warehouse, which may be stacked in one or more stacks.Such merchandise may be in containers, for example boxes, or may not.The merchandise may be associated with a value, for instance a monetaryvalue. A business may need to verify, inventory or account for coins,tokens or chips, each of which may be associated with some value, forinstance of monetary value. Such coins, tokens or chips, may be stackedin one or more piles or stacks.

Verification, inventorying or accounting may include determining whetherany objects are present, determining a total number of distinct objectsin a stack, determining a value of each object that is present and/ordetermining a cumulative value of the objects. Such is often performedby hand or by eye, with an individual manually counting the objects,visually assessing a value of each object, if any, and mentallyassessing a cumulative value of the objects.

Recently, some machine-vision based systems have been proposed toautomate the verification, inventorying or accounting functions. Many ofthe proposed machine-vision based systems have not been commerciallysuccessful. The lack of commercial success may be attributable to avariety of factors, for example high computational requirementsrequiring expensive processors and/or long processing times,inaccuracies, etc. New machine-vision based approaches to verifying,inventorying or accounting for stacks of objects are desirable.

BRIEF SUMMARY

A method of computationally processing images of areas which may containobjects may be summarized as including acquiring a pixelated color imageof an area as a set of pixilated color image data at least temporarilystored in at least one processor-readable medium; computationallypreprocessing by at least one processor the set of pixelated color imagedata for at least a portion of the area to produce a set of two colorcontour image data that represents at least the portion of the pixilatedcolor image, wherein a first color of the two color contour image datacorresponds to contour pixels and a second color of the two colorcontour image corresponds to non-contour pixels; computationallyprocessing by the at least one processor the set of two color contourimage data to find a location of a bottom of a stack of objects in theset of two color contour image data and to find a location of a top ofthe stack of objects in the set of two color contour image data, if anyobjects are in the at least portion of the area, wherein the stack ofobjects includes at least one object if any objects are in the at leastportion of the area; computationally determining by the at least oneprocessor a total number of objects in the stack, if any based at leastin part on a knowledge of the location of the bottom of the stack ofobjects and the location of the top of the stack of objects;computationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes; andcomputationally determining by the at least one processor a cumulativetotal value of the stack of objects based at least in part on the colorclass to which each of the objects in the stack of objects isclassified. Acquiring a pixilated color image of an area may includecapturing a color image by at least one image acquisition device of atleast a portion of a gaming table from a position spaced at a non-zerodegree and non-perpendicular angle out of a plane of a playing surfaceof the gaming table.

Computationally preprocessing by at least one processor the set ofpixelated color image data for at least a portion of the area to producea set of two color contour image data may include: comparing a set ofpixilated color image data representing a pixelated reference image ofthe area to the set of two color contour image data, said referenceimage having no stack of objects; and removing irregularities from theset of two color contour image data around a stack of objects in the twocolor contour image data based on the comparing.

The removing irregularities from the set of two color contour image datamay include: subtracting the set of pixilated color image datarepresenting a pixelated reference image of the area from the set of twocolor contour image data; and adding back to the set of two colorcontour image data an element of average background color of the area.

Computationally preprocessing by at least one processor the set ofpixelated color image data for at least a portion of the area to producea set of two color contour image data may include: color balancing a setof pixilated color image data representing a pixelated reference imageof the area to have it more closely match the set of two color contourimage data, said reference image having no stack of objects; rotatingthe set of two color contour image data to compensate for cameradistortion for one or more stacks of objects that are near edges of thetwo color contour image data; aligning the set of pixilated color imagedata representing the pixelated reference image and the set of two colorcontour image data to account for movement of an image acquisitiondevice used to acquire the pixelated color image; subtracting the set ofpixilated color image data representing a pixelated reference image ofthe area from the set of two color contour image data; and adding backto the set of two color contour image data image data representing anelement of average background color of the area; and either before thesubtracting or after the subtracting, converting the set of pixilatedcolor image data representing the pixelated reference image of the areaand the set of two color contour image data to gray scale if theconverting to gray scale had been determined likely to improve accuracyof the determining of the total number of objects in the stack.

Computationally preprocessing by at least one processor the set ofpixelated color image data for at least a portion of the area to producea set of two color contour image data may further include: eliminatingextraneous pixels from the set of two color contour image data that areoutside the at least portion of the area; and eliminating extraneouspixels that are not part of groups that at least approximately form theshape of a stack object based on a minimum threshold number of pixels.

The eliminating extraneous pixels that are not part of groups maycomprise: passing a template shaped as a stack object over the set oftwo color contour image data; counting a number of contour pixels withinan area defined by the template for each location of the template on theset of two color contour image data; and eliminating all pixels outsidethe area defined by the template at each location if the number ofcontour pixels within the area defined by the template contains theminimum threshold number of pixels.

The method of method of computationally processing images may furthercomprise: before computationally determining by the at least oneprocessor a total number of objects in the stack, determining ahorizontal center of the stack of objects based on an overallconcentration of contour pixels in the set of two color contour imagedata; and determining an estimated height of individual objects in thestack of objects based on a mean/average height of an individual objectof the stack rationalized for a distance between an image acquisitiondevice used to acquire the pixelated color image and an expectedlocation of the stack, said distance adjusted based on the location ofthe bottom of the stack at the determined horizontal center of the stackrelative to the expected location of the stack of objects.

The method of method of computationally processing images may furthercomprise reclassifying an object into a different color class based on adetermination that a color of the object is obscured by a shadow in thepixelated color image.

The reclassifying an object into a different color class may include:determining whether pixels of an object previously classified as blacksurpass a threshold regarding (HSB) values of the pixels; andreclassifying the object into a different color class if the thresholdis surpassed. The method of method of computationally processing imagesmay further comprise re-determining a value of the object based on thereclassification.

Computationally preprocessing by at least one processor the set ofpixelated color image data for at least a portion of the area to producea set of two color contour image data may include: converting at least aportion of the set of the pixelated color image data to a set of grayscale image data; and edge detection filtering the set of gray scaleimage data to produce a set of contour image data.

Computationally preprocessing by at least one processor the set ofpixelated color image data for at least a portion of the area to producea set of two color contour image data may further include: converting bythe at least one processor at least a portion of the set of contourimage data to a set of black and white contour image data.

Computationally preprocessing by at least one processor the set ofpixelated color image data for at least a portion of the area to producea set of two color contour image data may further include: eliminatingextraneous pixels from the set of two color contour image data.Converting by the at least one processor at least a portion of the setof contour image data to a set of black and white contour image data mayinclude: converting by the at least one processor a set of gray scalecontour image data to a set of hue, saturation, and brightness (HSB)contour image data; and converting by the at least one processor the setof HSB contour image data to the set of black and white contour imagedata.

Computationally preprocessing by at least one processor the set ofpixelated color image data for at least a portion of the area to producea set of two color contour image data may further include: logicallysegmenting the two color contour image data into a set of target pixelareas, each of the target pixel areas including a set of n by m pixels.

Computationally preprocessing by at least one processor the set ofpixelated color image data for at least a portion of the area to producea set of two color contour image data may further include eliminatingextraneous pixels from the set of two color contour image data by: foreach target pixel area of the set of target pixel areas, eliminating allcontour pixels in the target pixel area from the set of two colorcontour image data if a total number of the contour pixels in the targetpixel area is less that a threshold number of pixels per target pixelarea; and for each target pixel area of at least some of the set oftarget pixel areas, eliminating all contour pixels in the targetquadrant from the set of two color contour image data if the targetpixel area has less than a nearest neighboring threshold number ofnearest neighboring target pixel areas.

Computationally processing by the at least one processor the set of twocolor contour image data to find a location of a bottom and a top of astack of objects in the set of two color contour image data may include:applying a sample bounding box at successive intervals to groups oftarget pixel areas of the set of target pixel areas to find a locationof the bottom the stack of objects, each of the groups of target pixelareas including at least some target pixel areas with contour pixels ofthe set of two color contour image data that are below a determinedupper location in the set of two color contour image data; determining abottom quadrant to be the sample bounding box that includes the group oftarget pixel areas with the highest number of target pixel areas havingcontour pixels; determining whether a width of the target pixel areasincluded in the bottom quadrant fits a face ellipse of an object;determining whether a width of the target pixel areas included bottomquadrant fits a face ellipse of an object; determining that there is nostack present when the width does not fit the face ellipse; anddetermining a location of the bottom of the bottom quadrant when thewidth fits the face ellipse.

Computationally processing by the at least one processor the set of twocolor contour image data to find a location of a bottom and a top of astack of objects in the set of two color contour image data may include:determining a location of a bottom of the bottom quadrant by traversingdownwardly starting at a location offset from the determined upperlocation until a last one or more target pixel areas found within a leftand right boundary of the bottom quadrant.

Computationally processing by the at least one processor the set of twocolor contour image data to find a location of a bottom and a top of astack of objects in the set of two color contour image data may include:successively applying a sample bounding box to groups of target pixelareas of the set of target pixel areas at successive intervals from adetermined bottom of the stack of objects until a top streak of targetpixel areas contained within the sample bounding box is empty.

Computationally processing by the at least one processor the set of twocolor contour image data to find a location of a bottom and a top of astack of objects in the set of two color contour image data may include:determining a top bounding box based on the values of the samplebounding box with the sample bounding box positioned such that the topstreak of target quadrants contained within the sample bounding box isempty; successively applying the sample bounding box at successiveintervals to find a sample quadrant that fits an elliptical pattern;determining a list of coordinates of the target quadrants that intersecta top arc and a bottom arc of an imaginary ellipse the size of thesample bounding box; determining a total number of target quads thatoverlap the target quadrants of the imaginary ellipse from the samplebounding box; and determining the sample bounding box position with themost number of overlaps as the top quadrants.

Computationally processing by the at least one processor the set of twocolor contour image data to find a location of a bottom and a top of astack of objects in the set of two color contour image data may include:determining a total number of objects in the stack by determining adifference in the bottom of the top quadrant and a bottom of a bottomquadrant and a mean/average height of an individual object rationalizedfor a distance between the image acquisition device and the stack ofobjects.

Computationally processing by the at least one processor a portion ofthe set of pixelated color image data based on a knowledge of a locationof a bottom of the stack of objects and a location of a top of the stackof objects to respectively classify each object in the stack of objectsinto one of a defined set of color classes may include: for each of theobjects, classifying a number of pixels in a color pixel streak torespective hue, saturation, and brightness (HSB) values that representthe hue, saturation and brightness of the number of pixels, the colorpixel streak composed of a subset of the pixels between a top arc and abottom arc of an ellipse of the object.

Computationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes may furtherinclude: for each of the objects, classifying the number of pixels inthe color pixel streak to respective fundamental colors based on definedrelationships between HSB values and a defined set of fundamentalcolors.

Computationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes may furtherinclude: for each of the objects, determining whether the classifiedfundamental colors of the color pixel streak is in a set of color driftcolors.

Computationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes may furtherinclude: for each of the objects, determining whether the classifiedfundamental colors of the color pixel streak are a defined primaryfundamental color and a number of defined secondary fundamental colors.

Computationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes may furtherinclude: for each of the objects, if the classified fundamental colorsof the color pixel streak are not in the set of color drift colors andthe classified fundamental colors of the color pixel streak are not oneof the defined primary fundamental color or the defined secondaryfundamental colors, determining a fundamental color based on a mostcommon color shared by the pixels of the color pixel streak.

Computationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes may furtherinclude: for each of the objects, determining a value of the objectbased on a determined fundamental color and a defined set of fundamentalcolor to object value relationships.

Computationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes may furtherinclude: summing the determined values of each of the objects.

A system may be summarized as including at least one image capturingdevice configured to acquire a pixelated color image of an area as a setof pixelated color image data; and at least one object evaluationcomputing system having at least one processor and at least one storagemedium that at least temporarily stores a the set of pixelated colorimage data, the object evaluation computing system configured to:preprocess using the at least one processor the set of pixelated colorimage data for at least a portion of the area to produce a set of twocolor contour image data that represents at least the portion of thepixilated color image; process using the at least one processor the setof two color contour image data to find a location of a bottom of astack of objects in the set of two color contour image data and to finda location of a top of the stack of objects in the set of two colorcontour image data, if any objects are in the at least portion of thearea, wherein the stack of objects includes at least one object if anyobjects are in the at least portion of the area; determine using the atleast one processor a total number of objects in the stack, if any basedat least in part on a knowledge of the location of the bottom of thestack of objects and the location of the top of the stack of objects;process using the at least one processor a portion of the set ofpixelated color image data based on the knowledge of the location of thebottom of the stack of objects and the location of the top of the stackof objects to respectively classify each object in the stack of objectsinto one of a defined set of color classes; and determine using the atleast one processor a cumulative total value of the stack of objectsbased at least in part on the color class to which each of the objectsin the stack of objects is classified.

The at least one object evaluation computing system may be furtherconfigured to preprocess using the at least one processor the set ofpixelated color image data by: comparing a set of pixilated color imagedata representing a pixelated reference image of the area to the set oftwo color contour image data, said reference image having no stack ofobjects; and removing irregularities from the set of two color contourimage data around a stack of objects in the two color contour image databased on the comparing.

The removing irregularities may comprise: subtracting the set ofpixilated color image data representing a pixelated reference image ofthe area from the set of two color contour image data; and adding backto the set of two color contour image data an element of averagebackground color of the area.

The at least one object evaluation computing system may be furtherconfigured to preprocess using the at least one processor the set ofpixelated color image data by: color balancing a set of pixilated colorimage data representing a pixelated reference image of the area to haveit more closely match the set of two color contour image data, saidreference image having no stack of objects; rotating the set of twocolor contour image data to compensate for camera distortion for one ormore stacks of objects that are near edges of the two color contourimage data; aligning the set of pixilated color image data representingthe pixelated reference image and the set of two color contour imagedata to account for movement of an image acquisition device used toacquire the pixelated color image; subtracting the set of pixilatedcolor image data representing a pixelated reference image of the areafrom the set of two color contour image data; adding back to the set oftwo color contour image data image data representing an element ofaverage background color of the area; and either before the subtractingor after the subtracting, converting the set of pixilated color imagedata representing the pixelated reference image of the area and the setof two color contour image data to gray scale if the converting to grayscale had been determined likely to improve accuracy of the determiningof the total number of objects in the stack.

The at least one object evaluation computing system may be configured topreprocess using the at least one processor the set of pixelated colorimage data by: eliminating extraneous pixels from the set of two colorcontour image data that are outside the at least portion of the area;and eliminating extraneous pixels that are not part of groups that atleast approximately form the shape of a stack object based on a minimumthreshold number of pixels.

The eliminating extraneous pixels that are not part of groups maycomprise: passing a template shaped as a stack object over the set oftwo color contour image data; counting a number of contour pixels withinan area defined by the template for each location of the template on theset of two color contour image data; and eliminating all pixels outsidethe area defined by the template at each location if the number ofcontour pixels within the area defined by the template contains theminimum threshold number of pixels.

The at least one object evaluation computing system may be furtherconfigured to: before computationally determining using the at least oneprocessor a total number of objects in the stack, determine a horizontalcenter of the stack of objects based on an overall concentration ofcontour pixels in the set of two color contour image data; and determinean estimated height of individual objects in the stack of objects basedon a mean/average height of an individual object of the stackrationalized for a distance between an image acquisition device used toacquire the pixelated color image and an expected location of the stack,said distance adjusted based on the location of the bottom of the stackat the determined horizontal center of the stack relative to theexpected location of the stack of objects.

The at least one object evaluation computing system may be furtherconfigured to reclassify an object into a different color class based ona determination that a color of the object is obscured by a shadow inthe pixelated color image.

The at least one object evaluation computing system may be configured toreclassify an object by: determining whether pixels of an objectpreviously classified as black surpass a threshold regarding (HSB)values of the pixels; and reclassifying the object into a differentcolor class if the threshold is surpassed. The at least one objectevaluation computing system may be further configured to re-determine avalue of the object based on the reclassification.

The at least one object evaluation computing system may be furtherconfigured to convert at least a portion of the set of the pixelatedcolor image data to a set of gray scale image data and perform edgedetection filtering of the set of gray scale image data to produce a setof contour image data.

The at least one object evaluation computing system may be furtherconfigured to logically segment the two color contour image data into aset of target pixel areas, each of the target pixel areas including aset of n by m pixels.

The at least one object evaluation computing system may be furtherconfigured to eliminate extraneous pixels from the set of two colorcontour image data by: for each target pixel area of the set of targetpixel areas, eliminating all contour pixels in the target pixel areafrom the set of two color contour image data if a total number of thecontour pixels in the target pixel area is less that a threshold numberof pixels per target pixel area; and for each target pixel area of atleast some of the set of target pixel areas, eliminating all contourpixels in the target quadrant from the set of two color contour imagedata if the target pixel area has less than a nearest neighboringthreshold number of nearest neighboring target pixel areas.

The at least one object evaluation computing system may be furtherconfigured to: apply a sample bounding box at successive intervals togroups of target pixel areas of the set of target pixel areas to find alocation of the bottom the stack of objects, each of the groups oftarget pixel areas including at least some target pixel areas withcontour pixels of the set of two color contour image data that are belowa determined upper location in the set of two color contour image data;determine a bottom quadrant to be the sample bounding box that includesthe group of target pixel areas with the highest number of target pixelareas having contour pixels; determine whether a width of the targetpixel areas included in the bottom quadrant fits a face ellipse of anobject; determine whether a width of the target pixel areas includedbottom quadrant fits a face ellipse of an object; determine that thereis no stack present when the width does not fit the face ellipse; anddetermine a location of the bottom of the bottom quadrant when the widthfits the face ellipse.

The at least one object evaluation computing system may be furtherconfigured to: successively apply a sample bounding box to groups oftarget pixel areas of the set of target pixel areas at successiveintervals from a determined bottom of the stack of objects until a topstreak of target pixel areas contained within the sample bounding box isempty.

The at least one object evaluation computing system may be furtherconfigured to: determine a top bounding box based on the values of thesample bounding box with the sample bounding box positioned such thatthe top streak of target quadrants contained within the sample boundingbox is empty; successively apply the sample bounding box at successiveintervals to find a sample quadrant that fits an elliptical pattern;determine a list of coordinates of the target quadrants that intersect atop arc and a bottom arc of an imaginary ellipse the size of the samplebounding box; determine a total number of target quads that overlap thetarget quadrants of the imaginary ellipse from the sample bounding box;and determine the sample bounding box position with the most number ofoverlaps as the top quadrants.

The at least one object evaluation computing system may be furtherconfigured to: determine a total number of objects in the stack bydetermining a difference in the bottom of the top quadrant and a bottomof a bottom quadrant and a mean/average height of an individual objectrationalized for a distance between the image acquisition device and thestack of objects.

The at least one object evaluation computing system may be furtherconfigured to: classify for each of the objects a number of pixels in acolor pixel streak to respective hue, saturation, and brightness (HSB)values that represent the hue, saturation and brightness of the numberof pixels, the color pixel streak composed of a subset of the pixelsbetween a top arc and a bottom arc of an ellipse of the object.

The at least one object evaluation computing system may be furtherconfigured to: classify for each of the objects the number of pixels inthe color pixel streak to respective fundamental colors based on definedrelationships between HSB values and a defined set of fundamentalcolors.

The at least one object evaluation computing system may be furtherconfigured to: determine for each of the objects whether the classifiedfundamental colors of the color pixel streak are in a set of color driftcolors.

The at least one object evaluation computing system may be furtherconfigured to: determine for each of the objects a value of the objectbased on a determined fundamental color and a defined set of fundamentalcolor to object value relationships.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

In the drawings, identical reference numbers identify similar elementsor acts. The sizes and relative positions of elements in the drawingsare not necessarily drawn to scale. For example, the shapes of variouselements and angles are not drawn to scale, and some of these elementsare arbitrarily enlarged and positioned to improve drawing legibility.Further, the particular shapes of the elements as drawn, are notintended to convey any information regarding the actual shape of theparticular elements, and have been solely selected for ease ofrecognition in the drawings.

FIG. 1 is a schematic diagram of an environment including an objectevaluation system and a surface on which a number of objects arestacked, according to one illustrated embodiment.

FIG. 2 is a schematic diagram of an object evaluation computing system,according to one illustrated embodiment.

FIG. 3A is a representation of a digital image of a number of demarcatedareas and object stacks, according to one illustrated embodiment.

FIG. 3B is a representation of an image section of the digital image ofFIG. 3A, according to one illustrated embodiment.

FIGS. 4A and 4B are a representation of contour data of an imaged objectstack in a demarcated area corresponding to an image section, accordingto one illustrated embodiment.

FIG. 5 is a flow diagram of a method of evaluating objects in an image,according to one illustrated embodiment.

FIG. 6 is a flow diagram of a method of acquiring an image, according toone illustrated embodiment, which may be employed in the method of FIG.5.

FIG. 7A is a flow diagram of a method of preprocessing a color image,according to one illustrated embodiment, which may be employed inperforming the image preprocessing of the method of FIG. 5.

FIG. 7B is a flow diagram of a method of preprocessing a color imageincluding optionally performing reference image subtractionpre-processing, according to one illustrated embodiment, which may beemployed in performing the image preprocessing of the method of FIG. 5.

FIG. 7C is a flow diagram of a method of preprocessing a color imageincluding details of optionally performing reference image subtractionpre-processing, according to one illustrated embodiment, which may beemployed in performing the image preprocessing of the method of FIG. 7B.

FIG. 7D is a flow diagram of a method of preprocessing a color imageincluding further optional details of performing reference imagesubtraction pre-processing, according to one illustrated embodiment,which may be employed in performing the image preprocessing of themethod of FIG. 7B.

FIG. 8 is a flow diagram of a method of identifying contours in a grayscale image, according to one illustrated embodiment, which may beemployed in the method of FIG. 7A.

FIG. 9A is a flow diagram of a method of processing contours accordingto one illustrated embodiment, which may be employed in performing noiseelimination of the method of FIG. 7A.

FIG. 9B is a flow diagram of a method of processing contours accordingto another alternative illustrated embodiment, which may be employed inperforming noise elimination of the method of FIG. 7A.

FIG. 10A is a flow diagram of a method of locating a bottom of an objectstack, locating a top of the object stack, and determining a number ofobjects in the object stack according to one illustrated embodiment,which may be employed in performing image processing of the method ofFIG. 5.

FIG. 10B is a flow diagram of a method of determining a horizontalcenter of an object stack, locating a bottom of the object stack,locating a top of the object stack, and determining a number of objectsin the object stack according to one illustrated embodiment, which maybe employed in performing image processing of the method of FIG. 5.

FIG. 11 is a flow diagram of a method of locating a bottom of an objectstack according to one illustrated embodiment, which may be employed inthe method of FIG. 10A.

FIG. 12 is a flow diagram of a method of determining a bottom of abottom quadrant according to one illustrated embodiment, which may beemployed in the method of FIG. 11.

FIG. 13 is a flow diagram of a method of locating a top of an objectstack according to one illustrated embodiment, which may be employed inthe method of FIG. 10A.

FIG. 14 is a flow diagram of a method of determining a number of objectsin an object stack, which may be employed in the method of FIG. 10A00.

FIG. 15 is a flow diagram of a method of determining a cumulative valuefor objects in an acquired image according to one illustratedembodiment, which may be employed in performing the cumulative totaldetermination of the method of FIG. 5.

FIG. 16A is a flow diagram of a method of color classifying objects todetermine a value according to one illustrated embodiment, which may beemployed in performing the color classification of the method of FIG.15.

FIG. 16B is a flow diagram of a method of color classifying objects todetermine a value according to one illustrated embodiment, includingreclassifying particular objects obscured by shadow, which may beemployed in performing the color classification of the method of FIG.15.

DETAILED DESCRIPTION

In the following description, certain specific details are set forth inorder to provide a thorough understanding of various disclosedembodiments. However, one skilled in the relevant art will recognizethat embodiments may be practiced without one or more of these specificdetails, or with other methods, components, materials, etc. In otherinstances, well-known structures associated with computing systems,networks, and image acquisition have not been shown or described indetail to avoid unnecessarily obscuring descriptions of the embodiments.

Unless the context requires otherwise, throughout the specification andclaims which follow, the word “comprise” and variations thereof, suchas, “comprises” and “comprising” are to be construed in an open,inclusive sense, such as “including, but not limited to.”

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. Thus, the appearances of the phrases “in one embodiment” or“in an embodiment” in various places throughout this specification arenot necessarily all referring to the same embodiment. Furthermore, theparticular features, structures, or characteristics may be combined inany suitable manner in one or more embodiments.

As used in this specification and the appended claims, the singularforms “a,” “an,” and “the” include plural referents unless the contentclearly dictates otherwise. It should also be noted that the term “or”is generally employed in its sense including “and/or” unless the contentclearly dictates otherwise.

The headings and Abstract of the Disclosure provided herein are forconvenience only and do not interpret the scope or meaning of theembodiments.

FIG. 1 shows an environment 100, according to one illustratedembodiment, in which stacks of objects may be evaluated by an objectevaluation system.

In particular, stacks of objects 102 (e.g., 102 a-102 d) are arranged ona surface 104, for example a surface of a table 106. The surface 104 mayinclude demarcated areas 108 a-108 g (collectively 108) in which stacksof objects may be placed from time to time. The stacks of objects 102may include one or more objects that are generally arranged one on topof the other, such as in a uniform stack where the outside edges (i.e.,perimeters) of the objects align, or in an irregular stack where objectperimeters are unaligned. In this illustrated example, the stacks ofobjects 102 a, 102 b, 102 c, and 102 d are respectively located indemarcated areas 108 a, 108 c, 108 d, and 108 f.

Each of the objects has an associated value and/or worth that isvisually represented by the object's coloration. In some embodiments,the coloration of an object may be monochromatic, multicolored, and/ormay include markings of various shapes and/or patterns (e.g., on a faceof the object, on an outside perimeter of the object). In variousembodiments, the objects may include tokens, chips, coins, etc.

The environment 100 may be subject to fluctuating lighting conditions.For example, the lighting in the environment may vary due to a time ofday, artificial lighting, light emanating or reflecting from varioussources within the environment (e.g., televisions, illuminatedadvertisements, devices, etc.), shadows cast by objects and/or people inthe environment 100, etc. The lighting conditions may vary in luminance(e.g., brightness, darkness) and/or color. Due to such lightingvariations, an object's coloration may be subject to “color drift,” inwhich a color of an object may appear different under different lightingconditions. For example, a blue colored object may appear black underdarker light conditions, light blue under brighter light conditions, orpurple under a reddish light; a white object may appear yellowish undersome lighting conditions.

The environment 100 also includes an object evaluation system 110, whichincludes an image acquisition device 112 and an object evaluationcomputer system 114. The image acquisition device 112 may be any of avariety of image acquisition devices, such as including cameras, forinstance digital still cameras, video cameras, CMOS image sensors,and/or any other type of device for acquiring or capturing color images(including devices incorporating or paired with a frame grabber or thelike). The object evaluation computer system 114 may be a computingsystem or device that includes hardware, software, firmware, or somecombination thereof configured to perform appropriate portions of objectevaluation, such as shown and described in FIGS. 3A-3B, 4A-4B, and 5-16.Although the image acquisition device 112 is shown in FIG. 1 as beingattached to the object evaluation computer system 114, it will beappreciated that in other embodiments, the image acquisition device 112may be communicatively coupled to the object evaluation computer system114 in various other ways, such as via the network 116.

The image capture device 112 is positioned at a non-zero degree andnon-perpendicular angle out of a plane of the surface 104 to acquire orcapture an image of at least a portion of the surface 104 that includesone or more of the demarcated areas 108. For example, in someembodiments, the image acquisition device 112 is positioned at anelevated height above the surface 104 of the table 106 and oriented toacquire or capture images of demarcated areas 108, such that stacks ofone or more objects present within the demarcated areas appear in theimage with at least a portion of a top surface of the stack and a sidesurface of the stack are both visible in the image, such as shown anddescribed with respect to FIGS. 3A-3B. In some embodiments, the imageacquisition device 112 may be positioned below the surface 104 andoriented to acquire or capture images of one or more demarcated areas108 from below the surface 104, such as where the surface 104 istransparent or translucent to at least some wavelengths of light.

In one embodiment, the image acquisition device 112 is positioned andoriented to acquire or capture in a single image all the demarcatedareas 108 present on the surface 104 of the table 106. Positioning andorienting the image acquisition device may also include configuring theimage acquisition device 112 in various ways, such as specifying and/ordetermining a camera zoom (optical, digital), an image point (focus), anaperture setting, etc.

In various embodiments, the image acquisition device 112 may be affixedto the table 106, and/or incorporated into a device, article, orstructure (none of which are shown) that is affixed to placed on, orotherwise carried by the table 106. In some embodiments, the camera maybe remotely located from the table 106, such as being attached to astructure other than the table 106, a ceiling, held/worn by a human,placed underneath the table 106, etc. Furthermore, the object evaluationcomputer system 114 may be included in a device, article, or structurethat may be affixed to or placed on the table 106. In other embodiments,the object evaluation computer system 114 may be under the table 106, ormay be remotely located from the table 106. In some embodiments, theobject evaluation computer system 114 and the image acquisition device112 may be integrated into a single system, device, or casing.

As described in more detail herein, the object evaluation computersystem 114 of the object evaluation system 110 may execute instructionsstored on a computer readable medium (not shown in FIG. 1) so as toautomatically determine a value associated with a stack of one or moreobjects 102 present in a demarcated area 108. In particular, the objectevaluation system 110 may execute instructions to cause the imageacquisition device 112 to acquire a pixelated color image including oneor more of the demarcated areas 108, and process one or more sections ofthe acquired image respectively corresponding to the one or moredemarcated areas 108, so as to evaluate objects that are stacked in theone or more demarcated areas 108, if any. Such processing of an imagesection of the pixelated color image corresponding to a demarcated areamay include preprocessing the image section as a gray scale image toidentify contour data, processing the contour data to determine alocation of a top and a bottom of a stack of objects 102 located withina respective demarcated area 108, identifying a respective set of colorpixels associated with each of the objects in the stack from thepixelated color image based on the located top and bottom of the stack,determining a corresponding value for each of the objects in the stackbased on a color classification of the respective sets of color pixels,and determining a cumulative value of the stack of objects. As part ofthe color classification, the object evaluation computer system 114 mayaccount for color drift that may occur in the image, so as to determinean appropriate color classification for an object notwithstandingfluctuating lighting conditions in the environment 100. Thus, in atleast one embodiment, the object evaluation computer system 114 mayexecute instructions that cause the object evaluation computer system114 to determine the value associated with stacks of objects 102 presentin any one of the demarcated areas 108 of the environment 100 undervarious lighting conditions.

The object evaluation computer system 114 may be communicatively coupledto a communications network 116, such as to communicate with one or moreother computing systems and/or devices (not shown). For example, in someembodiments, the object evaluation computer system may provideinformation related to determined stack values to one or more computingsystems, such as a centralized server computer, to facilitate monitoringof the environment 100. In other embodiments, the object evaluationcomputer system 114 and the image acquisition device 112 may becommunicatively coupled via the network 116. The network 116 may take avariety of forms. For instance, the networks 116 may include wired,wireless, optical, or a combination of wired, wireless and/or opticalcommunications links. In addition, the network 108 may include publicnetworks, private networks, unsecured networks, secured networks orcombinations thereof, and may employ any one or more communicationsprotocols, for example TCP/IP protocol, UDP protocols, IEEE 802.11protocol, as well as other telecommunications or computer networkingprotocols.

Although only a single object evaluation system 110 is shown in theillustrated embodiment of FIG. 1, in other embodiments multiple objectevaluation systems 110 may be provided, such that each system determinesa value of stacks of objects 102 corresponding to a different one ormore of the demarcated areas 108 and/or corresponding to demarcatedareas corresponding to one or more other tables (not shown). Further,multiple image acquisition devices 112 may also be provided, such as foracquiring or capturing images of various of the demarcated areas 108 ofthe surface 104, or for acquiring or capturing images of demarcatedareas associated with different tables, with a single object evaluationcomputer system 114 processing images acquired or captured via multipleimage acquisition devices.

The described systems and techniques may be used in a variety ofcontexts. For example, in some embodiments, the described systems andtechniques may be used for monitoring various types of transactionsinvolving coins, tokens, and/or chips.

In at least one exemplary embodiment of the environment 100, the objectsthat are stacked are gambling chips that are wagered by players ofwagering games, such as games including, but not limited to, poker,blackjack, baccarat, roulette, wheel of fortune, etc. For example, suchan environment may include a casino, a gaming parlor, and/or any otherlocation in which wagering games are played. Each gambling chiprepresents a particular monetary denomination that may be wagered by aplayer, with each chip having a distinct coloration indicative of arespective monetary amount. For example, the following color anddenomination relationships may be typical in some embodiments: whiterepresenting $1, red representing $5, green representing $25, blackrepresenting $100, purple representing $500, and yellow representing$1000. In various embodiments, the coloration of a chip may be a singlecolor, multiple colors, and/or color patterns appearing on the chip.

In such an embodiment of the environment 100, the stacks of objects 102are bet stacks (e.g., stacks of one or more gambling chips), the table106 is a gaming table with playing surface 104, and the demarcated areas108 are betting circles. During a wagering game, the players placerespective bets by selecting a number of chips to wager and placing theselected chips in a respective betting circle of the gaming table. Thenumber of chips stacked in a betting circle constitutes a bet stack forthe respective player. Further, the number and denomination of the chipsin the bet stack represents a cumulative monetary amount that the playerwagers at a given time.

In such an embodiment, the object evaluation system 110 may be used tocapture and process one or more images of the surface of the gamingtable to evaluate bet stacks and determine monetary amounts that one ormore players are wagering. Such a system may be useful for a casino orother provider of the wagering game to monitor betting patterns of oneor more players, such as for the purposes of fraud detection (e.g.,detecting patterns consistent with card counting, etc.), to performgeneral tracking and/or accounting, etc.

The detailed structure and operation of the object evaluation system arediscussed in more detail below.

FIG. 2 and the following discussion provide a brief, general descriptionof an exemplary object evaluation computing system 200 in which thevarious illustrated embodiments can be implemented. The objectevaluation computing system 200 may, for example, implement the variousfunctions and operations discussed immediately above in reference toFIG. 1.

Although not required, some portion of the embodiments will be describedin the general context of computer-executable instructions or logic,such as program application modules, objects, or macros being executedby a computer. Those skilled in the relevant art will appreciate thatthe illustrated embodiments as well as other embodiments can bepracticed with other computer system configurations, including handhelddevices for instance Web enabled cellular phones or PDAs, multiprocessorsystems, microprocessor-based or programmable consumer electronics,personal computers (“PCs”), network PCs, minicomputers, mainframecomputers, and the like. The embodiments can be practiced in distributedcomputing environments where tasks or modules are performed by remoteprocessing devices, which are linked through a communications network.In a distributed computing environment, program modules may be locatedin both local and remote memory storage devices.

The object evaluation computing system 200 may include one or moreobject evaluation computers 204 (only one illustrated in FIG. 2). Theobject evaluation computer(s) 204 may take the form of a conventionalPC, server, or other computing system executing instructions. The objectevaluation computer 204 includes a processing unit 206, a system memory208 and a system bus 210 that couples various system componentsincluding the system memory 208 to the processing unit 206. The objectevaluation computer 204 will at times be referred to in the singularherein, but this is not intended to limit the embodiments to a singlesystem, since in certain embodiments, there will be more than one systemor other networked computing device involved. Non-limiting examples ofcommercially available systems include, but are not limited to, an 80×86or Pentium series microprocessor from Intel Corporation, U.S.A., aPowerPC microprocessor from IBM, a Sparc microprocessor from SunMicrosystems, Inc., a PA-RISC series microprocessor from Hewlett-PackardCompany, or a 68xxx series microprocessor from Motorola Corporation.

The processing unit 206 may be any logic processing unit, such as one ormore central processing units (CPUs), microprocessors, digital signalprocessors (DSPs), application-specific integrated circuits (ASICs),field programmable gate arrays (FPGAs), etc. Unless described otherwise,the construction and operation of the various blocks shown in FIG. 2 areof conventional design. As a result, such blocks need not be describedin further detail herein, as they will be understood by those skilled inthe relevant art.

The system bus 210 can employ any known bus structures or architectures,including a memory bus with memory controller, a peripheral bus, and alocal bus. The system memory 208 includes read-only memory (“ROM”) 212and random access memory (“RAM”) 214. A basic input/output system(“BIOS”) 216, which can form part of the ROM 212, contains basicroutines that help transfer information between elements within theobject evaluation computer 204, such as during start-up. Someembodiments may employ separate buses for data, instructions and power.

The object evaluation computer 204 also includes a hard disk drive 218for reading from and writing to a hard disk 220, and an optical diskdrive 222 and a magnetic disk drive 224 for reading from and writing toremovable optical disks 226 and magnetic disks 228, respectively. Theoptical disk 226 can be a CD or a DVD, while the magnetic disk 228 canbe a magnetic floppy disk or diskette. The hard disk drive 218, opticaldisk drive 222 and magnetic disk drive 224 communicate with theprocessing unit 206 via the system bus 210. The hard disk drive 218,optical disk drive 222 and magnetic disk drive 224 may includeinterfaces or controllers (not shown) coupled between such drives andthe system bus 210, as is known by those skilled in the relevant art.The drives 218, 222, 224, and their associated computer-readable media220, 226, 228, provide nonvolatile storage of computer-readableinstructions, data structures, program modules and other data for theobject evaluation computer 204. Those skilled in the relevant art willappreciate that other types of computer-readable media may be employedto store data accessible by a computer, such as magnetic cassettes,flash memory cards, Bernoulli cartridges, RAMs, ROMs, smart cards, etc.

Program modules can be stored in the system memory 208, such as anoperating system 230, one or more application programs 232, otherprograms or modules 234, drivers 236 and program data 238.

The application programs 232 may, for example, include image acquisitionlogic 232 a, image preprocessing logic 232 b, locate object logic 232 c,and color classification logic 232 d. The logic 232 a-232 d may, forexample, be stored as one or more executable instructions.

As discussed in more detail below, the image acquisition logic 232 a mayinclude logic or instructions to acquire one or more digital colorimages of a surface having one or more stacks of objects thereon, suchas a surface 104 of the table 106 (FIG. 1), via the image acquisitiondevice 262. Such digital color images may be stored at least temporarilyin system memory 208 or another computer readable medium for processingas described below. The image acquisition logic 232 a may also identifyor otherwise isolate a section of an acquired or captured image thatcorresponds to a particular demarcated area that may contain a stack ofobjects, such as a demarcated area 108. As discussed in more detailbelow, the image preprocessing logic 232 b may include logic orinstructions to generate two-color contour data from a section of theacquired or captured digital color image, such that the two-colorcontour data may be further processed to evaluate a stack of one or moreobjects that may be present in the acquired or captured image. Suchpreprocessing may include subdividing the two-color contour data into aset of target pixel areas associated with the image section. Asdiscussed in more detail below, the locate object logic 232 c mayinclude logic or instructions to process a set of target pixel areasassociated with the image section to identify various features of astack of objects, such as to identify a top and a bottom of the stack,and a number of objects present in the stack. Additionally, as discussedin more detail below, the color classifier logic 232 d may include logicor instructions to identify a set of color pixels associated with eachof the one or more objects in the stack, automatically determine a colorclassification for each of the objects, and determine a cumulative valuefor the stack based on the color classifications of each object. Therelationship between an object's color classification and a value may bebased on a known/configured relationship between colors and associatedvalues. Furthermore, the color classification logic may compensate forvarious lighting conditions in the environment 100, so as to allowobjects imaged in the acquired digital color image to be appropriatelyclassified even in an environment with fluctuating lighting conditionsin which color drift may occur. Such application programs 232 mayexecute the logic 232 a-232 d as methods or processes set out in thevarious flow charts discussed below.

The system memory 208 may also include other programs/modules 234, suchas including logic for calibrating and/or otherwise training variousaspects of an object evaluation system 200, such as logic that may beused to configure the location of demarcated areas within images, theexpected size of objects imaged within demarcated areas, color/valuerelationship, etc. The other programs/modules 234 may include variousother logic for performing various other operations and/or tasks.

The system memory 208 may also include communications programs 240 thatpermit the object evaluation computer 204 to access and exchange datawith other systems or components, such as with the image acquisitiondevice 262 and/or optionally with one or more other computer systems anddevices (not shown).

While shown in FIG. 2 as being stored in the system memory 208, theoperating system 230, application programs 232, other programs/modules234, drivers 236, program data 238 and communications 240 can be storedon the hard disk 220 of the hard disk drive 218, the optical disk 226 ofthe optical disk drive 222 and/or the magnetic disk 228 of the magneticdisk drive 224.

A user can enter commands and information into the object evaluationcomputer 204 through input devices such as a touch screen or keyboard242, a pointing device such as a mouse 244, a push button (not shown),and/or foot-control (not shown). Other input devices can include amicrophone, joystick, game pad, tablet, scanner, biometric scanningdevice, etc. These and other input devices are connected to theprocessing unit 206 through an interface 246 such as a universal serialbus (“USB”) interface that couples to the system bus 210, although otherinterfaces such as a parallel port, a game port or a wireless interfaceor a serial port may be used. A monitor 248 or other display device iscoupled to the system bus 210 via a video interface 250, such as a videoadapter.

The image acquisition device 262 may be a video camera, digital stillcamera, or other imaging device. In some embodiments, the imageacquisition device may include a frame grabber or other device forcapturing, buffering, converting, and/or otherwise obtaining digitalcolor images for processing by the object evaluation computer 204. Inthis illustrated embodiment, the image acquisition device 262 is shownas being connected to interface 246, although in other embodiments, theimage acquisition device 262 may be connected to the object evaluationcomputer 204 using other means, such as via the network 214, via one ormore intermediary devices (not shown), etc.

Furthermore, although not illustrated here, the object evaluationcomputer 204 may include a triggering device and/or other mechanism formanually triggering the object evaluation computer 204 to acquire orcapture an image via the image acquisition device 262, such as a pushbutton, a foot control, etc. In some embodiments, the triggering deviceand/or other mechanism may be attached to or otherwise coupled to theimage acquisition device 262. Further, image acquisition may beautomatically triggered by the object evaluation computer 204, such asbased on a timer and/or an occurrence of another event, and/or may beremotely triggered via the network 214.

In some embodiments, the object evaluation computer 204 operates in anenvironment 100 (FIG. 1) using one or more of the logical connections tooptionally communicate with one or more remote computers, servers and/orother devices via one or more communications channels, for example, oneor more networks such as the network 214. These logical connections mayfacilitate any known method of permitting computers to communicate, suchas through one or more LANs and/or WANs. Such networking environmentsare well known in wired and wireless enterprise-wide computer networks,intranets, extranets, and the Internet. For example, in someembodiments, the object evaluation computer 204 may communicate with oneor more additional image acquisition devices (not shown) via the network214, such as for the purposes of acquiring or capturing images of othersurfaces and/or object stacks for processing by the object evaluationcomputer 204. In some further environments, the object evaluationcomputer 204 may communicate with one more other computing systems ordevice, such as to receive indications to acquire or capture and/orevaluate images, and/or to provide information regarding evaluatedimages, such as information regarding the determined value of objects,etc.

In some embodiments, the network interface 256, which is communicativelylinked to the system bus 210, may be used for establishingcommunications over the network 214. Further, the database interface252, which is communicatively linked to the system bus 210, may be usedfor establishing communications with a database stored on one or morecomputer-readable media 260. For example, such a database 260 mayinclude a repository for storing information regarding objectevaluation, color tables, object color-value relationship information,etc. In some embodiments, the database interface 252 may communicatewith the database 260 via the network 214.

In an environment 100 (FIG. 1), program modules, application programs,or data, or portions thereof, can be stored in another server computingsystem (not shown). Those skilled in the relevant art will recognizethat the network connections shown in FIG. 2 are only some examples ofways of establishing communications between computers, and otherconnections may be used, including wirelessly. In some embodiments,program modules, application programs, or data, or portions thereof, caneven be stored in other computer systems or other devices (not shown).

For convenience, the processing unit 206, system memory 208, networkport 256 and interfaces 246, 252 are illustrated as communicativelycoupled to each other via the system bus 210, thereby providingconnectivity between the above-described components. In alternativeembodiments of the object evaluation computer 204, the above-describedcomponents may be communicatively coupled in a different manner thanillustrated in FIG. 2. For example, one or more of the above-describedcomponents may be directly coupled to other components, or may becoupled to each other, via intermediary components (not shown). In someembodiments, system bus 210 is omitted and the components are coupleddirectly to each other using suitable connections.

In many of the embodiments presented below, the objects that comprise astack are illustrated and described as having a cylindrical shape with acircular periphery or peripheral edge, and in particular, the objectsare chips. Furthermore, the demarcated areas in which the objects arestacked are illustrated and described as having a circular shape. Itwill be appreciated that in other embodiments, the objects and/or thedemarcated areas may have different corresponding shapes.

FIG. 3A is a representation of a pixelated image 300. FIG. 3B shows animage section 304 c of the pixelated image 300. FIGS. 3A-3B providecontext for description of embodiments of an object evaluation routinedescribed with respect to FIGS. 5-16. Although FIGS. 3A-3B are notillustrated in color, it will be appreciated that the pixelated image300 is representative of a digital color image that comprises of pixeldata having color components (e.g., red, green, blue), with various ofthe objects appearing in the image 300 having corresponding colorinformation that is represented in the pixel data of image 300.

FIG. 3A is a representation of a pixelated color image 300 of thesurface 104, including demarcated areas 108 and object stacks 102,according to one illustrated embodiment. The image 300 may be acquiredor captured by the image acquisition device 112 (FIG. 1), such as forprocessing by the object evaluation computing system 114 (FIG. 1) toevaluate one or more stacks of objects presented in the image. In thisexample of the pixelated image 300, each of the stacks of objects 102 islocated within a respective demarcated area 108, with stack 102 a havingfive objects, stack 102 b having two objects, stack 102 c having threeobjects, and stack 102 d having one object, although other stack sizesare possible and anticipated. Although this illustrated embodiment showsseven demarcated areas 108, other embodiments may include more or lessdemarcated areas.

As is shown in FIG. 3A, the image 300 has been acquired or captured byan image acquisition device 112 from such a position and orientationthat each of the imaged stacks of objects 102 has a top surface and aportion of a perimeter of the stack showing. For example, with respectto the stack of objects 102 c, a top surface 310 and the side perimeter312 of the stack 102 c are visible in the image. In at least oneembodiment, where the objects are cylindrical (e.g., chips), the topsurface of the stack appears in the image as an ellipse, such asillustrated here. In addition, in this illustrated embodiment, thedemarcated areas 108 are circular, and thus also appear as ellipses inthe images.

Various image sections 304 a-304 g (collectively 304) of the image 300are shown in FIG. 3A. Each of the image sections 304 corresponds to aparticular demarcated area 108, and in particular, defines a section ofthe image 300 (e.g., a group of pixels) that is to be processed forobject evaluation in the corresponding demarcated area. For example, theimage area 304 a defines a section of the image 300 that is to beprocessed for evaluating a stack of one or more objects present (if any)in the demarcated area 108 a. Image sections 304 b-304 g respectivelycorrespond to demarcated areas 108 b-108 g. Each of the image sections304 includes an area of the image 300 that substantially encompasses arespective demarcated area 108 and a stack of one or more objects 102that may be present in the respective demarcated area. In someembodiments, the image sections 304 may be defined by a user during amanual configuration process.

In various embodiments, the image 300 may be comprised of variousamounts of digital image data (e.g., pixel data). As one example, theimage 300 may correspond to a 3.1 megapixel image, which isapproximately comprised of 2048×1536 pixels, although other image sizesmay be used (e.g., 2 megapixel, 8 megapixel, 10 megapixel, etc.). As istypical in digital image processing, the upper-left corner of the image300 is illustrated as corresponding to the origin of the image (pixelposition X=0, Y=0), with the X-axis being the horizontal axis andincreasing towards the right of the illustrated digital image 300, andthe Y-axis being the vertical axis and increasing towards the bottom ofthe illustrated digital image 300.

Further, each of the image sections 304 may correspond to a particularsubset of the image pixels of the image 300, and may be defined based onpixel coordinates corresponding to the particular subsets. For example,in this illustrated embodiment, each of the image sections may bedefined as having an origin pixel (e.g., the upper-left corner)corresponding to pixel coordinates of the image 300, and a specifiedheight and width.

Of course, it will be appreciated that other embodiments are possible.For example, the image 300 may have more or less image data, and/orother coordinate systems may be used. Furthermore, the sizes and shapesof the image sections 304 may vary in different embodiments, and evenfrom one image section to another image section. In addition, in someembodiments, prior to processing an image section 304 for evaluation bythe object evaluation system, the image section 304 may be cropped fromthe image 300 and interpolated into an image having larger dimensions(e.g., a larger number of pixels), such as by digitally zooming theimage section 304.

FIG. 3B shows a representation of the image section 304 c of the image300 (FIG. 3A). The upper-left corner of the image section 304 ccorresponds to pixel position (x₀, y₀) from the image 300, whichcorresponds to the pixel position of the upper-left corner of the imagesection 304 c illustrated in FIG. 3A. In particular, in FIG. 3B theimage section 304 c includes the object stack 102 b present in thedemarcated area 108 c. The top and bottom of the demarcated area areindicated respectively as 354 and 356. In this illustrated example, theobject stack 102 b includes two objects, with a chip 364 being at thetop of the stack 102 b, and a chip 366 being at the bottom of the stack102 b. A top surface 362 (also referred to herein as a “face,” an“object face,” or a “chip face”) of the top chip 364 is visible, as wellas a portion of a perimeter of the top chip 364 and a portion of aperimeter of the bottom chip 366. In this embodiment, when the stack 102b is positioned within the demarcated area 108 c, the bottom object 366of the stack 102 b will appear in the image section 304 c as beingsubstantially within the demarcated area 108 c.

A bounding box 370 is illustrated in FIG. 3B. The bounding box 370 isrepresentative of the size of an object face as it may appear in theimage section 304 c. In particular, the bounding box 370 is a rectanglethat closely encompasses the elliptical object face as it may appear inimage section 304 c, with the height and width of the rectanglerespectively defined by the vertical height (in pixels) and thehorizontal width (in pixels) of the elliptical object face. As describedin more detail elsewhere, the bounding box may be used by the objectevaluation system as part of locating objects and/or identify variousfeatures of objects and/or object stacks.

In some embodiments, the bounding box 370 may be defined based on animage of a representative object (e.g., a chip) as it appears whenplaced within the demarcated area 108 c, such that the object iscompletely encompassed within the demarcated area 108 c when imaged. Forexample, in some embodiments, the bounding box may be predefined duringa calibration process, in which a user places a representative objectwithin the demarcated area 108 c, the object is imaged, and the boundingbox is determined based on the resulting image (e.g., automaticallyand/or manually).

Indication 372 shows a height of a single object (e.g., the height ofthe bottom object 366) as it appears in the image section 304 c. Theheight of the object as it appears in the image section may bepredefined, such as based on a single object as it appears in the imagesection when the object positioned within the demarcated area (e.g.,such as during a calibration/training process). In some embodiments, theheight of the object may be a determined mean/average height of objectsas they appear within in the image section.

Although FIGS. 3A-3B and 4A-4B are depicted as having aligned objectstacks, it will be appreciated that the techniques described herein areapplicable in cases where object stacks are skewed or otherwiseirregularly stacked.

FIGS. 4A and 4B show a representation of a set of target pixel areas 402associated with an image section of the pixelated image 300, accordingto one illustrated embodiment. FIGS. 4A and 4B provide context fordescription of embodiments of an object evaluation routine describedwith respect to FIGS. 5-16.

In particular, in FIG. 4A, the set of target pixel areas 402 logicallysubdivides at least a portion of the associated image section into agrid of target pixel areas, with each target pixel area corresponding toa particular n by m pixel area of the image section—target pixel areasare variously referred to as “target quadrants” or “target quads” insome embodiments described herein. In this sense, the term quadrant orquad is not intended to be limiting to an area divided into foursections or sub-areas, but rather is intended to be broad referring toany division of an area without regard to the total number of sectionsor sub-areas. In some embodiments, the size of the n by m target pixelarea may be configurable, such as based on the lighting conditions inthe imaging environment.

In one embodiment, the set of target pixel areas spans the entireportion of the image section that may be considered as part of locatinga stack of one or more objects (e.g., identifying a top and/or bottom ofthe stack) positioned within the demarcated area in the associated imagesection. For example, in a typical embodiment, the set of target pixelareas may span from the leftmost visible portion of the demarcated areato the rightmost visible portion of the demarcated area (e.g., column420 corresponds to the leftmost portion of the demarcated area, andcolumn 422 corresponds to the rightmost portion of the demarcated area),and from some position above the demarcated area that may correspond toa highest considered elevation of a stack of objects present in thedemarcated area to the bottommost visible portion of the demarcated area(e.g., row 424 corresponds to the highest considered elevation, and row426 corresponds to the bottommost visible portion of the demarcatedarea).

In at least one embodiment, the target pixel areas of the set of targetpixel areas 402 may be 10×10 pixel areas. For example, in such anembodiment, the target pixel area 402 a corresponds to a pixel area thatincludes pixel coordinates (x₀, y₀) to (x₀+9, y₀+9) of the associatedimage section, while a target pixel area 402 b corresponds to a pixelarea that includes pixel coordinates (x₀+10, y₀) to (x₀+19, y₀+9), and atarget pixel area 402 c corresponds to a pixel area covering pixelcoordinates (x₀, y₀+10) to (x₀+9, y₀+19), and so on. In this illustratedembodiment, each target pixel area has up to eight neighboring targetpixel areas (e.g., above, below, right, left, upper-left, upper-right,lower-left, lower-right). For example, the eight target pixel areas 416surrounding the target pixel area 402 d are neighboring target pixelareas of target pixel are 402 d.

As is described in more detail herein, the object evaluation systempreprocesses pixels of an image section to determine which pixels of theimage section correspond to contours (e.g., edges). For example, eachpixel of the image section may be read along with its eight neighboringpixels (e.g., a 3×3 pixel kernel) to determine whether the pixelcorresponds to contour, such as by evaluating the contrast betweenvarious of the neighboring pixels. Each pixel that is identified as acontour pixel is associated with an appropriate target pixel area of theset of target pixel areas 402 that corresponds to the identified pixel'scoordinates, such as based on the identified pixel's coordinates beingwithin the target pixel area. After the contour pixels are subdividedinto the set of target pixel areas 402 in such a manner, the objectevaluation system may then utilize the set of target pixel areas 402 aspart of locating a stack of one or more objects and identifying imagedata corresponding to each object in the stack, such as locating the topand bottom of the stack of objects. In some embodiments, the objectevaluation system may use the location of the top and bottom todetermine a total number of chips in the stack and to locate, for eachof the one or more objects in the stack, an area of the image sectionthat corresponds to at least a portion of a perimeter of the object,such as the perimeter area 412 corresponding to the top object and theperimeter area 414 corresponding to the bottom object.

In this illustrated embodiment of FIG. 4A, the set of target pixel areas402 is associated with the image section 304 c (FIG. 3A-3B) whichincludes the object stack 102 b (with objects 364 and 366) and thedemarcated area 108 c. In particular, the preprocessing of the imagesection 304 c may detect contours associated with the objects 364 and366 (FIG. 3B), such as the edge 404 corresponding to a face ellipse ofthe top object 364, the edge 405 corresponding to a portion of a bottomellipse of the top object and/or a portion of a top ellipse of thebottom object, the edge 406 corresponding to a portion of a bottomellipse of the bottom object 366, and edges 407 corresponding to thesides of the objects, and associated with the demarcated area 108 c(FIG. 3A-3B), such as the edge 407. In some embodiments, the top of thestack of objects 102 b imaged in the image section 304 c corresponds tothe edge 404 and the bottom of the stack of objects corresponds to edge406.

Each pixel of the image section 304 c that is determined to correspondto a contour is assigned into a corresponding target pixel area in theset of target pixel areas 402. For the purposes of illustration, it maybe assumed that contours are detected corresponding to each of the edges404-408. As such, any of the target pixel areas that are intersected byone of the edges 404-408 include contour pixels corresponding to aportion of the edge that intersects the target pixel area. For example,the target pixel area 402 d includes contour pixels corresponding to theportion of the edge 408 that intersects the target pixel area 402 d.

Target pixel areas that do not include any detected contour pixels areconsidered “empty.” For example, target pixel area 402 b does notinclude any edge data, nor does the group of target pixel areas 410, andare thus considered empty target pixel areas. In some embodiments, agroup of neighboring target pixel areas that do not include anyassociated edge data, such as group 410, are referred to as a streak ofempty target pixel areas.

It will be appreciated that the edges 404-407 are merely exemplary, andin real-world applications, additional edges may be identified in animage section, such as corresponding to various markings and/or visibledesigns included on the objects, the surface, or within the demarcatedarea; and/or other items/information that appears within the imagesection (e.g., objects or items in the background of the image, etc.);etc. In addition, edges and/or portions of edges associated with one ormore objects may be missing, such as if there is insufficient image datato detect an edge or a portion of an edge (e.g., when an edge is inshadow, in dark lighting, obscured). Even in such situations, thetechniques described herein may be utilized to evaluate a stack ofobjects.

In FIG. 4B, an embodiment of a sample bounding box 418 being applied toa group of target pixel areas of the set of target pixel areas 402 isillustrated. The sample bounding box 418 has a height and width thatcorresponds to the height and width of an object face as it appears inan image section (e.g., such as the calibrated bounding box 370 of FIG.3B). Another embodiment of a sample bounding box 419 is also beingapplied to another group of target pixel areas. The sample bounding box419 has a height that corresponds to the height of the demarcated area(e.g., from the top to the bottom of the demarcated area) as it appearsin the image section and a width that corresponds to the width of theobject face. The sample bounding boxes 418 and 419 may be used to definegroups of target pixel areas of the set of target pixel areas 402, suchas for the purposes of locating and/or identifying one or more featuresof a stack of objects that may be present in the imaged demarcated area,such as described in more detail elsewhere.

Furthermore, although the embodiment of FIGS. 4A and 4B described aboveuse 10×10 pixel target pixel areas, the size of target pixel areas maybe smaller or larger in other embodiments. In some embodiments, the sizeof a target pixel area may be determined based on the lightingconditions in the imaging environment, as low light images may tend tohave low contrast which may result in blurred edges, while bright lightimages may tend to have high contrast which may result in sharper edges.For example, in low light conditions, a larger target pixel area may bepreferable (e.g., 10×10 and greater), while in brighter light conditionsa smaller target pixel area may be preferable. In addition, although theset of target areas 402 is illustrated as subdividing an entire imagesection, in other embodiments, the set of target areas may insteadcorrespond to one or more subsections of the image section.

FIG. 5 shows a method 500 of operating at least one object evaluationcomputer system (e.g., system 114 and/or 204 of FIGS. 1 and 2,respectively) to facilitate evaluating a stack of objects in an image,according to one illustrated embodiment.

At 502, the method may acquire or capture a pixelated color image of asurface of a table, the pixelated image including at least onedemarcated area which may contain a stack of one or more objects.

At 504, the method may preprocess at least a section of the acquiredcolor image that corresponds to the at least one demarcated area toproduce a set of two color contour image data for the section of theacquired color image. In particular, the two color contour image dataincludes pixels having a first color that corresponds to contour pixelsand pixels having a second color that corresponds to non-contour pixels.For example, in some embodiments, white pixels correspond to contourpixels and black pixels correspond to non-contour pixels. In addition,in some embodiments, the at least a section of the acquired color imagemay include a sub-portion of the acquired color image, while in otherembodiments, the at least a portion may include the entire acquiredcolor image.

At 506, the method may process the set of two-color contour data tolocate a stack of one or more objects within the demarcated area, ifsuch a stack is present. In particular, as part of such processing, themethod may identify from the two color contour data a location of a topof the stack and a location of a bottom of the stack within the imagesection, as well as determine how many objects are present in the stack.

At 508, the method may determine a cumulative total value for the stackof objects present in the demarcated area, if any. As part ofdetermining a cumulative total value, the method may identify a set ofcolor pixels for each of the objects that are present in the stack, suchas based on the determined location of the top and bottom of the stack,and classify each of the objects in the stack into an appropriate colorclassification based on a respective identified set of color pixel. Themethod may then determine a value associated with each of the colorclassifications, and sum each of the determined values to calculate thecumulative total value for the stack.

Although not illustrated in the method 500, in other embodiments, theacquired or captured pixelated color image may include multipledemarcated areas that may each contain a respective stack of one or moreobjects. In some such embodiments, the method 500 may identify multipleimage sections of the acquired image, with each of the identified imagesections corresponding respectively to a particular demarcated area ofthe multiple demarcated areas, and process each of the image sections toevaluate a respective stack of objects that may be present in the imagesection, such as by performing steps 504-508 for each of the multipleimage sections.

FIG. 6 shows a method 600 of acquiring or capturing an image, accordingto one illustrated embodiment. The method 600 may be employed inperforming the image acquisition of method 500 (FIG. 5).

At 602, an image acquisition device may acquire or capture a pixelatedcolor image of one or more demarcated areas on a surface of a table. Theimage acquisition device is positioned at non-zero degree andnon-perpendicular angle out from the surface of the table and orientedto acquire or capture at least a top surface of a stack of objectslocated in any of the one or more demarcated areas as well as a portionof a side perimeter of the stack, such as described elsewhere.

FIG. 7A shows a method 700 of preprocessing at least a section of anacquired or captured color image, according to one illustratedembodiment, which may be employed in performing the image preprocessingof the method 500 (FIG. 5).

At 702, the method may generate a gray scale image from the section ofthe acquired or captured color image. The gray scale conversion may beperformed using any number of known techniques. For example, in someembodiments, the method may convert each color pixel of the section togray scale by estimating the luminance of the color components (e.g.,red, green, and blue) of the pixel and using the estimated luminance todetermine a gray scale value for the pixel.

At 704, the method may analyze the gray scale image data to identifycontours and produce a set of two color contour data. Contours may beidentified by applying an edge detection filter to the gray scale image,with the two color image data produced by associating edge pixels with afirst color and non-edge pixels with a second color. In at least oneembodiment, edge pixels may be associated with white pixels, whilenon-edges may be associated with black pixels. Other colors may, ofcourse, be used in other embodiments.

At 706, the method may eliminate noise from the two color contour imagedata. For example, in some embodiments, the method may eliminateextraneous edge pixels from the two color contour image data, such asedge pixels that are unlikely to correspond to edges of the stack of oneor more objects, etc.

FIG. 7B shows a method 708 of preprocessing a color image, according toone illustrated embodiment, which may be employed in performing theimage preprocessing of the method of FIG. 5.

At 710, the method may optionally perform reference image subtractionpre-processing such that markings or irregularities in or on the surfaceon which the stack of objects is resting may be subtracted or reducedfrom the pixilated data. This may be performed by comparing a set ofpixilated color image data representing the pixelated reference image tothe set of two color contour image data of the acquired pixelated colorimage including the stack of objects. For example, the reference imagemay be an image of the area on which the stack of objects is resting,but not including the stack of objects. This image may then besubtracted from the acquired or captured color image including the stackof objects, thereby leaving the stack of objects itself as thedifference between the images. Further details of such a process aredescribed with reference to FIG. 7C and FIG. 7D below.

At 712, the method may optionally generate a gray scale image from thesection of the acquired or captured color image resulting from thesubtraction above. The gray scale conversion may be performed using anynumber of known techniques. For example, in some embodiments, the methodmay convert each color pixel of the section to gray scale by estimatingthe luminance of the color components (e.g., red, green, and blue) ofthe pixel and using the estimated luminance to determine a gray scalevalue for the pixel.

At 714, the method may analyze the gray scale image data to identifycontours and produce a set of two color contour data. Contours may beidentified by applying an edge detection filter to the gray scale image,with the two color image data produced by associating edge pixels with afirst color and non-edge pixels with a second color. In at least oneembodiment, edge pixels may be associated with white pixels, whilenon-edges may be associated with black pixels. Other colors may, ofcourse, be used in other embodiments.

At 716, the method may eliminate noise from the two color contour imagedata. For example, in some embodiments, the method may eliminateextraneous edge pixels from the two color contour image data, such asedge pixels that are unlikely to correspond to edges of the stack of oneor more objects, etc.

FIG. 7C shows a method 718 of preprocessing a color image includingdetails of optionally performing reference image subtractionpre-processing, according to one illustrated embodiment, which may beemployed in performing the image preprocessing of the method of FIG. 7B.In particular, FIG. 7C shows in further details the reference imagesubtraction pre-processing and other details which may be performedbefore block 712 of FIG. 7B.

At 720, the method optionally color balances the image to have it moreclosely match a reference image taken without the stack of objects. Forexample, this may include color balancing the set of pixilated colorimage data representing the pixelated reference image having no stack ofobjects in order to have it more closely match the set of two colorcontour image data of the acquired pixelated color image that includesthe stack of objects.

At 722, the method optionally rotates the image to compensate for cameradistortion for stacks that are near the edges of the image. In referenceto the present example, the set of two color contour image data may berotated to compensate for camera distortion for one or more stacks ofobjects that are near edges of the two color contour image data.

At 724, the method may align the image and reference image (alsoreferred to as image registration) to account for camera movement. Inreference to the present example, the method may align the set ofpixilated color image data representing the pixelated reference imageand the set of two color contour image data to account for movement ofan image acquisition device used to acquire the pixelated color image.

At 726, the method may then subtract the reference image from the image.In reference to the present example, the method may subtract the set ofpixilated color image data representing the pixelated reference image ofthe area from the set of two color contour image data. This mayeliminate many table markings which would interfere with later steps.

At 728, the method may add back an element of average background colorof the original image to the image resulting from the subtraction aboveof the reference image. Regarding the present example, the method mayadd back to the set of two color contour image data resulting from thesubtraction of the reference image data, image data representing anelement of average background color of the area on which the stack ofobjects is resting.

FIG. 7D shows a method 730 of preprocessing a color image includingfurther optional details of performing reference image subtractionpre-processing, according to one illustrated embodiment, which may beemployed in performing the image preprocessing of the method of FIG. 7B.In particular, FIG. 7C shows in further details the reference imagesubtraction pre-processing and other details which may be performedbefore block 712 of FIG. 7B.

At 732, the method optionally color balances the image to have it moreclosely match a reference image taken without the stack of objects. Forexample, this may include color balancing the set of pixilated colorimage data representing the pixelated reference image having no stack ofobjects in order to have it more closely match the set of two colorcontour image data of the acquired pixelated color image that includesthe stack of objects.

At 734, the method optionally rotates the image to compensate for cameradistortion for stacks that are near the edges of the image. In referenceto the present example, the set of two color contour image data may berotated to compensate for camera distortion for one or more stacks ofobjects that are near edges of the two color contour image data.

At 736, the method may align the image and reference image to accountfor camera movement (also referred to as image registration). Inreference to the present example, the method may align the set ofpixilated color image data representing the pixelated reference imageand the set of two color contour image data to account for movement ofan image acquisition device used to acquire the pixelated color image.

At 738, the method may optionally convert the color image and colorreference image to grayscale. In reference to the present example, themethod may optionally convert the set of pixilated color image datarepresenting the pixelated reference image data to gray scale andconvert the set of two color contour image data to gray scale if theconverting to gray scale had been. The determination of whether toconvert the data to gray scale may be based on whether it is likely toimprove accuracy of the determining of the total number of objects inthe stack. Also, the conversion may occur either before the subtractingdescribed below of the set of pixilated color image data representingthe pixelated reference image or after the subtracting described belowof the set of pixilated color image data.

At 740, the method may then subtract the reference image from the image.In reference to the present example, the method may subtract the set ofpixilated color image data representing the pixelated reference image ofthe area from the set of two color contour image data. This mayeliminate many table markings which would interfere with later steps.

At 742, the method may add back an element of average background color(or gray scale value) of the original image to the image resulting fromthe subtraction above of the reference image. Regarding the presentexample, the method may add back to the set of two color contour imagedata resulting from the subtraction of the reference image data, imagedata representing an element of average background color (or gray scalevalue) of the area on which the stack of objects is resting.

FIG. 8 shows a method 800 of identifying contours in a gray scale image,according to one illustrated embodiment, which may be employed in themethod 700 (FIG. 7A) as part of preprocessing an image section.

At 802, the method may apply an edge detection filter to the gray scaleimage. For example, in some embodiments, the method may read in pixelsfrom the gray scale image and output a gray scale pixel value for eachpixel indicative of a correspondence between the pixel and an edge. In atypical edge detection filter, the output gray scale pixel values rangefrom black (indicating no or a low correspondence to an edge) to white(indicating a high correspondence to an edge), although other colorranges are possible. Various edge detection techniques may be used invarious embodiments, such as including, but not limited to, RobertsCross, Sobel, Canny, and/or other techniques. In at least oneembodiment, the method may read a sample pixel and its eight neighboringpixels (e.g., a 3×3 pixel kernel), compare HSB values of various of theneighboring pixels (e.g., by subtracting various of the neighboringpixels from each other), and determine a gray scale pixel value for thesample pixel based on a determined contrast between the variousneighboring pixels.

At 804, the method may determine HSB (hue, saturation, and brightness)values for each of the gray scale pixels output from the edge detectionfilter.

At 806, the method may classify each of the gray scale pixels as beingeither black or white based on the determined HSB value for the pixel,with white indicating that the pixel is an edge pixel and blackindicating that the pixel is not an edge pixel (or at least not an edgepixel of interest). In some embodiments, the pixels may be classified aseither black or white based on the intensity of one or more of their HSBvalues with respect to a range of HSB values. For example, the methodmay classify pixels having HSB brightness values within a lower range(e.g., 0 to 65) as being black, and may classify pixels having HSBbrightness values within a higher range (e.g., 66-100) as being white.

At 808, the method may assign each pixel that is classified at 806 asbeing white to a corresponding target pixel area of a set of targetpixel areas associated with the image section. As previously describedwith respect to FIG. 4, the set of target pixel areas may be organizedas a grid of pixel areas that logically overlays/subdivides the imagesection. Pixels are assigned to a corresponding target pixel area of theset of target pixel areas based on the pixel's coordinates being withinthe area covered by the target pixel area. Thus, in this embodiment, theset of target pixel areas only includes pixels that are determined tocorrespond to edges, which in this example embodiment are pixels thatare classified as being white. Of course, the method could operate onthe reverse image, classifying the set of target pixels as being black,or some other color.

FIG. 9A shows a method 900 of eliminating noise from two color contourimage data, according to one illustrated embodiment, which may beemployed in the method 700 (FIG. 7A). In particular, the method 900 maybe performed after the method 800 (FIG. 8) has subdivided the two color(e.g., black and white) edge data into a set of target pixel areasassociated with an image section. In some embodiments, eliminating noisefrom the two color contour data may remove at least some of the edgepixels (white pixels) from the two color contour data that are notassociated with a stack of objects captured in the image section, so asto minimize the amount of edges associated with image data that is notrelated to the stack of one or more objects, such as edges associatedwith the demarcated area. In this illustrated embodiment, the noiseelimination method 900 may reduce the number of target pixel areas ofthe set of target pixel areas that have associated white pixels.

At 902, the method may remove white pixels from the two color contourimage data if the white pixels are assigned to a target pixel area thatcontains less than a threshold number of white pixels, essentiallymaking such target pixel areas empty. As one illustrative example, in anembodiment using a 10×10 target pixel area, the threshold may be threepixels.

At 904, the method may further remove white pixels from the two colorcontour image data if the white pixels are assigned to target pixelareas that do not have a threshold number of neighboring target pixelareas that contain white pixels. As previously noted, in someembodiments each target pixel area may have up to eight neighboringtarget pixel areas. In some such embodiments, the threshold number ofneighboring target pixel areas may be three. In other embodiments, theneighboring target pixel areas may also include target pixel areas thatare within a close proximity to the target pixel area being evaluated,but are not necessarily direct neighbors.

The threshold values used at 902 and 904 may vary from embodiment toembodiment. In some embodiments, the threshold values may be adjusteddepending on the lighting conditions in the imaging environment.

FIG. 9B shows a method 906 of eliminating noise from two color contourimage data, according to another alternative illustrated embodiment,which may be employed in the method 700 (FIG. 7A). In particular, themethod 906 may be performed after the method 800 (FIG. 8) has subdividedthe two color (e.g., black and white) edge data into a set of targetpixel areas associated with an image section.

At 908, the method may eliminate pixels that are outside of and belowthe demarcated area (e.g., the demarcated area 108 c of FIG. 3B). Inreference to the present example, the method may eliminate extraneouspixels from the set of two color contour image data that are outside thedemarcated area.

At 910, pixels may be eliminated that are not part of groups that atleast approximately form the shape of a stack object based on a minimumthreshold number of pixels. In particular, one way this may beaccomplished is by passing a template shaped as a stack object over theset of two color contour image data and then counting a number ofcontour pixels within an area defined by the template for each locationof the template on the set of two color contour image data. All pixelsmay then be eliminated that are outside the area defined by the templateat each location if the number of contour pixels within the area definedby the template contains the minimum threshold number of pixels.

FIG. 10A shows a method 1000 of processing image data corresponding toan image section to locate a stack of objects within the demarcated areacorresponding to the image, according to one illustrated embodiment,which may be employed in performing processing 506 in the method 500(FIG. 5).

At 1002, the method may search for a bottom of the stack of objectslocated in the demarcated area, such as by analyzing two color contourdata associated with the image section to identify contour data thatlikely corresponds to the bottom of the stack of objects. For example,in some embodiments, the method may attempt to locate contour data thatcorresponds to a size and/or shape of an object in a stack as it islikely to appear in the image section when positioned within thedemarcated area. If the bottom of the stack is not located, then themethod may determine that there is no stack of objects within thedemarcated area, at which point the method may end or return to processother potential image sections corresponding to other demarcated areasand/or acquire a new image of the demarcated area. As used herein and inthe claims, a stack may include one or more objects, and where multipleobjects those objects are carried or placed one on top the other,whether aligned or misaligned.

At 1004, if the bottom of the stack has been located within thedemarcated area at 1002, the method may search for a top of the stack ofobjects, such as by analyzing the two color contour data to identifycontour data that likely corresponds to the top of the stack of objects.For example, in some embodiments, the method may locate a top of a stackby analyzing the two color contour data to find a contour correspondingto a face ellipse of an object in the stack. In some embodiments, themethod may search for the top of the stack based on knowledge of thelocation of the bottom of the stack determined at 1002. For example, themethod may search for a face ellipse in an area of the two color contourdata that is likely to correspond to an object being positioned abovethe determined bottom of the stack and below a streak of empty targetpixel areas (e.g., streak 410 of FIG. 4A).

At 1006, the method determines the number of objects in the object stackbased on the determined location of the top and bottom of the stack ofobjects.

FIG. 10B shows a method 1008 of determining a horizontal center of anobject stack, locating a bottom of the object stack, locating a top ofthe object stack, and determining a number of objects in the objectstack according to one illustrated embodiment, which may be employed inperforming image processing of the method of FIG. 5.

At 1010, the method may determine the horizontal center of the stackbased on the overall concentration of remaining contours/pixels in theimage. In reference to the present example, the method may determine ahorizontal center of the stack of objects based on an overallconcentration of contour pixels in the set of two color contour imagedata.

At 1012, the method may search for a bottom of the stack of objectslocated in the demarcated area, such as by analyzing two color contourdata associated with the image section to identify contour data thatlikely corresponds to the bottom of the stack of objects. For example,in some embodiments, the method may attempt to locate contour data thatcorresponds to a size and/or shape of an object in a stack as it islikely to appear in the image section when positioned within thedemarcated area. If the bottom of the stack is not located, then themethod may determine that there is no stack of objects within thedemarcated area, at which point the method may end or return to processother potential image sections corresponding to other demarcated areasand/or acquire a new image of the demarcated area. As used herein and inthe claims, a stack may include one or more objects, and where multipleobjects those objects are carried or placed one on top the other,whether aligned or misaligned.

At 1014, if the bottom of the stack has been located within thedemarcated area at 1012, the method may search for a top of the stack ofobjects, such as by analyzing the two color contour data to identifycontour data that likely corresponds to the top of the stack of objects.For example, in some embodiments, the method may locate a top of a stackby analyzing the two color contour data to find a contour correspondingto a face ellipse of an object in the stack. In some embodiments, themethod may search for the top of the stack based on knowledge of thelocation of the bottom of the stack determined at 1002 and/or thehorizontal center of the stack at 1010. For example, the method maysearch for a face ellipse in an area of the two color contour data thatis likely to correspond to an object being positioned above thedetermined bottom of the stack and below a streak of empty target pixelareas (e.g., streak 410 of FIG. 4A).

At 1016, the method may adjust the expected height of stack objectsbased on location of the bottom of stack. In particular, the method maydetermine this based on a mean/average height of an individual object ofthe stack rationalized for a distance between an image acquisitiondevice used to acquire the pixelated color image and an expectedlocation of the stack. The distance between an image acquisition deviceused to acquire the pixelated color image and an expected location ofthe stack may be adjusted based on the location of the bottom of thestack at the determined horizontal center of the stack relative to theexpected location of the stack of objects (e.g., an expected location ofthe stack of objects in the demarcated area.

At 1018, the method may determine the number of objects in the objectstack based on the determined location of the top and bottom of thestack of objects and the adjusted expected height of stack objects.

FIG. 11 shows a method 1100 of locating a bottom of an object stackwithin a demarcated area according to one illustrated embodiment, whichmay be employed in the method 1000 (FIG. 10A). In particular, in thisembodiment, the method 1100 processes a set of target pixel areasassociated with an image section to locate a bottom of a stack ofobjects, such as the set of target pixel areas generated by method 800(FIG. 8) and filtered by the noise elimination method 900 (FIG. 9A).

At 1102, the method may apply a sample bounding box to each of multiplegroups of target pixel areas of the set of target pixel areas that arelocated below the top of the demarcated area. The method may use thesample bounding box to identify candidate groups of target pixel areasbelow the top of the demarcated area that potentially correspond to thebottom of a stack of objects. In a typical embodiment, the top of thedemarcated area corresponds to the topmost portion of the demarcatedarea as it appears in the image section (e.g., the smallest Y pixelcoordinate of the demarcated area in the image). Furthermore, the samplebounding box may be defined to have a height that corresponds to theheight of the demarcated area as it appears in the image and a widththat corresponds to a known width of a face of an object as it appearswhen positioned within the demarcated area (e.g., box 419 of FIG. 4B).In particular, at 1102, the method may iteratively apply the samplebounding box to groups of target pixel areas by traversing the set oftarget pixel areas below the demarcated area from left to right, so asto define candidate groups of target pixel areas within the entiredemarcated area below the top of the demarcated area.

At 1104, the method may determine which of the candidate groups oftarget pixel areas defined by the sample bounding box has the largestnumber of target pixel areas that include white pixels (e.g., edgedata).

At 1106, the method may define the bottom quadrant to be the samplebounding box that includes the group of target pixel areas that has thelargest number of target pixel areas with white pixels (or the lowestnumber of empty target pixel areas). The “bottom quadrant” is a samplebounding box that encompasses the group of target pixel areas thatpotentially includes contour data corresponding to the bottom of a stackof objects present within the demarcated area (if any).

At 1108, the method may determine the width of the bottom quadrant, suchas by subtracting the X coordinate of rightmost target pixel area withinthe bottom quadrant from the X coordinate of the leftmost target pixelarea within the bottom quadrant.

At 1110, the method may determine whether the determined width fitsdimensions of a known face ellipse of the face of an object as itappears when positioned within the demarcated area (e.g., such as acalibrated face ellipse). For example, the method may compare the widthdetermined at 1108 to a known width of the face ellipse (and/or thewidth of the sample bounding box). If the result of the comparison isequal and/or within some tolerance (e.g., within 1 or 2 target pixelareas), then the width determined at 1108 is determined to fit the faceellipse. In some embodiments, the tolerance may be configured based onlighting conditions in the environment, such as during a calibrationprocess.

If at 1110 the determined width does not fit the face ellipse, themethod may determine at 1112 that no stack of objects is present withinthe demarcated area.

If instead at 1110 it is determined that the width does fit the faceellipse, the method may determine at 1114 the bottom of the bottomquadrant, which is determined to be the bottom of the stack of objects.

At 1116 the method 1100 may end or terminate.

In some cases, such as if the bottom of the stack is not located, themethod may exit and or return to method 500 (FIG. 5) to acquire orcapture another image of the demarcated area and/or process anotherimage section corresponding to another demarcated area. In otherembodiments, such as when a bottom of the stack is located, the method1100 may simply return or otherwise end such that further processing ofthe image section may occur (e.g., to locating a top of the stack,perform color classification, etc.).

It will be appreciated that in other embodiments, the sample boundingbox used in the method 1100 may be of different dimensions. For example,in one embodiment, the sample bounding box may have dimensions thatcorrespond to the size of a calibrated object face, such as samplebounding box 418 (FIG. 4B), and in such cases may be iteratively appliedto target pixel areas below the target quadrant to locate a group oftarget quadrants corresponding to a bottom of the stack (e.g., to definethe bottom quadrant). In addition, in some embodiments, the width of thebottom quadrant may be determined at 1108 by considering target pixelareas that are located outside of the bottom quadrant, such as targetpixel areas that are within a defined proximity of the bottom quadrant.

FIG. 12 shows a method 1200 of determining a bottom of a bottom quadrantor area, according to one embodiment, which may be employed in themethod 1100 (FIG. 11).

At 1202, the method may traverse downwardly (in the positive Ydirection), starting halfway down from the top of the demarcated area(halfway between the top and bottom of the demarcated area as it appearsin the image section), until a last target pixel area that is confinedwithin the left and right coordinates of the of the bottom quadrant orarea is found. The last target pixel area refers to one or more targetpixel areas of the set of target pixel areas that include white pixelsand are closest to the bottom of the demarcated area (e.g., have thehighest Y coordinate position) within the left and right boundaries ofthe bottom quadrant. The bottom of the bottom quadrant may then be setto be a value corresponding to the last target pixel area, e.g., a Ycoordinate of the last target pixel area, and/or a Y coordinate of oneor more white pixels associated with the last target pixel area.

In some embodiments, the method may not necessarily start from halfwaybetween the top and bottom of the demarcated area. For example, in someembodiments, the method may start at the top of the bottom quadrantand/or from some other position within the demarcated area correspondingto a minimum Y position at which the bottom of a stack of objects mayappear in the image section and still be considered to be within thedemarcated area. Such is possible since, based on the angle at which theimage is acquired or captured, it may be certain that a front edge(i.e., edge closest to the image acquisition device) of a bottommostobject in a stack contained within a demarcated area on the surface willappear at least halfway between the top and the bottom of the demarcatedarea. Such assumption may advantageously speed up processing.Furthermore, in other embodiments, the bottom of the stack may insteadbe found by locating an elliptical contour at a maximum Y positionwithin the demarcated area that matches the elliptical pattern of atleast a bottom portion of a calibrated elliptical object face.

FIG. 13 shows a method 1300 of locating a top of an object stackaccording to one illustrated embodiment, which may be employed in themethod 1000 (FIG. 10A). In particular, in this embodiment, the method1300 may process a set of target pixel areas associated with the imagesection to locate a top of a stack of objects, such as after the method1100 (FIG. 11) has been performed to locate the bottom quadrant.

At 1302, the method may position the sample bounding box at the bottomof the bottom quadrant of the set of target pixel areas, the samplebounding box having a width and height corresponding to the width andheight of a known face ellipse of an object face (e.g., box 418 of FIG.4B). For example, the method may set the sample bounding box such thatthe bottom of the sample bounding box aligns with the bottom of thebottom quadrant and the left and right of the sample bounding box alignrespectively with the left and right of the bottom quadrant.

At 1304, the method may determine if the top row of target pixel areaswithin the sample bounding box contains a streak of empty target pixelareas, such as where all the target pixel areas in the top row areempty. In other embodiments, a top streak of empty target pixel areasmay include less than the entire top row, such as a minimum definedstreak length. It is noted that depending on where the stack of objectsis positioned within the demarcated area and/or depending on the numberof the objects in the stack, the top streak of empty target pixel areasmay be located above the demarcated area.

At 1306, if a top streak of empty target pixel areas was not found at1304, the method may iterate the sample bounding box up the set oftarget pixel areas (in the negative Y direction) until a top streak ofempty target pixel areas is located within the sample bounding box or,in some cases, the top of the image section is reached. For example, insome embodiments the sample bounding box may be iterated up by a singlerow of target pixel areas.

At 1308, after the streak of empty target pixel areas is found, themethod may set a top quadrant to be the sample bounding box thatincludes the determined top row of empty target pixel areas. The topquadrant is a sample bounding box that encompasses a group of targetpixel areas that potentially corresponds to the top object in a stack,and will eventually define the location of target pixel areas that aredetermined to correspond to a face of the top object in the stack.

At 1312, the method may iterate the sample bounding box downward (in thepositive Y direction), such as by a single row of target pixel areas.

At 1314, the method may determine a list of coordinates of target pixelareas that intersect a top arc and a bottom arc of an imaginary ellipsedefined by the sample bounding box. In some embodiments, the imaginaryellipse defined by the sample bounding box has a height and width thatcorresponds to the height and width of the sample bounding box (e.g.,and thus may correspond to an ellipse having a height and width of aface ellipse of a calibrated object face). The list of coordinates mayindicate which of the target pixel areas of the set of target pixelareas that fall within the sample bounding box intersect the top andbottom arcs of the imaginary ellipse. It will be appreciated that inother embodiments, the objects may be of other shapes and/or have faceswith shapes other than ellipses (e.g., rectangular, pentagonal,octagonal, etc.), and in such embodiments, the sample bounding box maybe used to define an imaginary shape corresponding to the appropriateshape of the face of such objects.

At 1316, the method may determines the number of the target pixel areasindicated by the list of coordinates that have white pixels, and thusthe number of target pixel area within the sample bounding box thatcontain edge data that overlaps with the top and bottom arc of theimaginary ellipse.

At 1318, the method may determine a sample bounding box that has thelargest number of overlaps. For example, the method may iteratively movethe sample bounding box from the sample bounding box set at 1312downward (in a positive Y direction) until the bottom of the samplebounding box is positioned at the bottom of the bottom quadrant,determine the number of target areas that intersect the imaginaryellipse at each iteration of the sample bounding box, and identify thesample bounding box that has the largest determined number of overlaps.The sample bounding box that includes the most overlaps is assumed to bethe top of the object stack.

At 1320, the method may set the top quadrant to be the determinedsampled bounding box that has the largest number of overlaps.

In some embodiments, if the method 1300 is unable to find a samplebounding box that includes at least a minimum number of overlaps (e.g.,90%), then the method may indicate that no stack has been found.Furthermore, in some embodiments, the sample bounding box may also beiterated from left to right, such as within some limited proximity ofthe left and right of the bottom quadrant, to locate a group of targetquadrants that corresponds to the face ellipse. In addition, in someembodiments, the sample bounding box and/or the imaginary ellipse may beexpanded and/or contracted as part of locating a group of targetquadrants that corresponds to the face ellipse.

FIG. 14 shows a method 1400 of determining a number of objects in anobject stack, which may be employed in the method of FIG. 10A00. Inparticular, in this embodiment, the method 1400 may be performed aftermethods 1100 (FIG. 11), 1200 (FIGS. 12), and 1300 (FIG. 13) have beenperformed to locate the bottom quadrant and the top quadrant.

At 1402, the method may subtract the bottom of the top quadrant from thetop of the bottom quadrant. In some embodiments, the bottom of the topquadrant may be a value that corresponds to a Y position of one or moretarget pixel areas that are located at the bottom of the top quadrant,and/or to a position of one or more white pixels associated with one ormore target pixel areas located at the bottom of the top quadrant. Thebottom of the bottom quadrant may be a value that corresponds to a Yposition as discussed with respect to the method 1200 (FIG. 12). Inparticular, the bottom of the top quadrant corresponds to a location ofthe bottom of a face ellipse of an object positioned at the top of thestack, and the bottom of the bottom quadrant corresponds to a locationof the bottom edge (e.g., bottom edge ellipse) of an object positionedat the bottom of the stack within the demarcated area.

At 1404, the method divides the subtracted value by a height of anindividual object as the object may appear in the image section (e.g.,the mean/average height of the object accounting for distance andorientation with respect to the image acquisition device). The result ofthe division is determined to be the number of objects in the stack.

FIG. 15 shows a method 1500 of determining a cumulative value forobjects in an acquired image according to one illustrated embodiment,which may be employed in performing the method 500 (FIG. 5).

At 1502, the method may determine a color classification for each of theobjects in the stack. In particular, the method may identify colorpixels from the acquired or captured pixelated color image thatcorrespond to each of the objects in the stack, and classify the colorof each of the objects based on the identified pixels. The pixels may beidentified based on a determined knowledge of the location of the topand bottom of the object stack, such as locations determined in themethod 1000 (FIG. 10A).

At 1504, the method may determine a value of each of the objects basedon their respective color classification. In some embodiments,relationships between color classifications and respective values may beconfigured or otherwise trained and stored in memory (e.g., such as in atable or other data structure), such that the method may retrieve avalue associated with a particular color classification.

At 1506, the method sums all of the determined values for each object todetermine the total/cumulative value of the objects in the stack.

At 1508, the method returns the sum.

FIG. 16A shows a method 1600 of color classifying objects anddetermining a value for objects based on color classifications,according to one illustrated embodiment, which may be employed inperforming the method 1500 (FIG. 15).

At 1602, the method may set an iterator I to 1.

At 1604, the method may set a maximum value J to be the number ofobjects in the stack, such as the number of objects determined in method1400 (FIG. 14).

At 1606, the method may acquire or capture a set of color pixels fromthe acquired pixelated color image as a color pixel streak thatcorresponds to an object I. The color pixel streak is a group of colorpixels that corresponds to at least a portion of a perimeter area of theobject I as imaged in the color image.

The color pixel streak may be acquired or captured for the object Ibased on knowledge of the top of the stack and bottom of the stack. Forexample, in some embodiments, the method may assume that each of theobjects in a stack of objects has elliptical perimeter arcs associatedwith a top and bottom of the object's visible perimeter (as it appearsin the image) that are similar in shape to an elliptical arc of the faceellipse. Thus, after the top quadrant has been located, the method mayderive and/or otherwise approximate the locations of the tops andbottoms of an object's visible perimeters. For example, in someembodiments, the method may determine how the face ellipse appears inthe image (e.g., such as based on a three-dimensional perspective of theobject face), and then may determine the shape of the top and bottomarcs of the object with respect to the face ellipse and where the topand bottom perimeter arcs for the object appears in the image relativeto the face ellipse (e.g., such as based on the known and/or determinedimage height of an object with respect to the determined location of thetop of the stack). The location of the top and bottom arc for each Ithimage may be determined based an iterated offset (in the positive Ydirection) from the location of the top face ellipse.

After the top and bottom arcs corresponding to the perimeter of the Ithobject have been determined, the method may select a portion of thecolor pixels from the pixelated color image located within the areabetween the top and bottom arcs. In some embodiments, the method mayselect a portion of the color pixels that corresponds to approximately30-40% of the width of the object and approximately 90% of thecalibrated height of an object, although in other embodiments, themethod may select more or less.

At 1608, the method converts the pixels in the color pixel streak intotheir corresponding HSB values (e.g., using a known technique to convertRGB components of a pixel into HSB values).

At 1610, the method may classify the pixels in the streak intofundamental colors based on the HSB values of the pixels. For example,in some embodiments, a predefined mapping table may be used to map HSBvalues to corresponding fundamental colors. Fundamental colors, alsoreferred to as “culture colors,” may include a set of color categories,such as including red, blue, green, white, black, purple, pink, yellow,brown, gray, and cyan, to which the various HSB values in the streak maybe mapped. The term fundamental colors is not intended to be coextensivewith the concept of primary colors, but rather is a subset of colorswhich are distinctly recognized across a variety of human cultures,hence the denomination as culture colors. In some embodiments, thefundamental colors included in the set may be limited to include onlythose colors that may actually occur in the objects that may be stackedin the demarcated area.

At 1611, the method may determine whether the pixels in the pixel streakhave the same color. If so, the method may determine at 1616 that thefundamental color associated with the Ith object is found. If not, themethod may continue to 1612.

At 1612, the method may determine whether the colors in the pixel streakintersect a color in a color-drift table. The color drift table mayinclude mappings between the fundamental colors and a set of (or subsetof) the possible colors that the fundamental color may drift to undervarious lighting conditions. If the colors included in the pixel streakis located in the color drift table (e.g., intersect the colors in thetable), then the table may identify a corresponding fundamental colorbased on the drift. The mappings may be predefined mappings and/orotherwise trained/calibrated.

If, at 1612, the colors in the pixel streak intersect the color drifttable, then the method may determine at 1616 that the fundamental colorassociated with the Ith object is found. If instead at 1612, it isdetermined that two or more fundamental colors are possibly present inthe pixel streak, the routine may continue 1614.

At 1614, the method may determine whether the colors in the pixel streakintersect a primary fundamental color and a top one or more secondaryfundamental colors. As used herein and in the claim, the term primary isused to refer to a first, most likely or dominate one of the fundamentalcolors, and is not intended to refer to the concept of primary colors.For example, in some embodiments, the primary fundamental color may bethe fundamental color that has the highest ration of correspondingpixels in the streak, and the top secondary fundamental colors are thetop most represented fundamental colors in the streak other than theprimary fundamental color.

At 1614, the method may determine the primary and top one or more (e.g.,three) secondary fundamental colors present in the streak, then may usea lookup table to determine a single resulting fundamental color thatthe pixel streak corresponds to based on the determined primary and topone or more secondary fundamental colors. The lookup table may, forexample, include mappings between combinations of primary and topsecondary colors and corresponding resulting fundamental colors. In someembodiments, the lookup table may include ratios of primary and topsecondary fundamental colors and a correspondence between such ratiosand a resultant fundamental color. The lookup table may be pre-populatedwith such mappings and/or may be populated with such mappings based onrun-time heuristics.

If, at 1614, the colors intersect a primary fundamental color and a topthree secondary fundamental colors, then, at 1616, the fundamental colorassociated with the Ith object is found.

Otherwise, at 1618, the method may determine the fundamental color thatcorresponds to the highest number of pixels in the pixel streak as beingthe fundamental color for the object.

At 1620, the method may determine a value of the object based on atrained or otherwise configured relationship between values andfundamental colors. For example, in some embodiments, the relationshipmay be maintained in a memory (e.g., such as in a table, map, or otherdata structure), such that a value of the object may be identified basedon the determined fundamental color for the object.

At 1622, the method determines whether the Ith object is the last object(e.g., whether I is equal to J). If not, I may be incremented at 1624,and the method may return to 1606 to process the next object. If so, themethod may continue to 1626 to return the determined values for each ofthe objects in the stack.

FIG. 16B shows a method 1628 of color classifying objects to determine avalue according to one illustrated embodiment, including reclassifyingparticular objects obscured by shadow, which may be employed inperforming the color classification of the method of FIG. 15. FIG. 16Bis similar to that of FIG. 16A. However, FIG. 16B includes theadditional blocks 1621 a and 1621 b.

At 1621 a the method may determine whether an object previouslyclassified as black surpasses threshold. In particular, the method maydetermine whether pixels of an object previously classified as blacksurpass a threshold regarding particular (HSB) values of the pixels,which may indicate the object is obscured by a shadow in the pixelatedcolor image. The particular threshold may vary depending on the desiredsensitivity to detect a shadow in the image.

At 1621 b, the method may reclassify the object and re-determine thevalue of the object. In particular, the method may reclassify the objectinto a different color class if the threshold is surpassed andre-determine a value of the object based on the reclassification.

It will be appreciated that in some embodiments, the objects in thestack may be multi-colored, such that the perimeters of the objectsinclude multiple colors. As such, the pixel streak may include pixelscorresponding to the multiple colors present on the perimeters. In somesuch embodiments, the method 1600 may identify multiple colors for theobject and determine a value for the object based on the identifiedmultiple colors.

The above description of illustrated embodiments, including what isdescribed in the Abstract, is not intended to be exhaustive or to limitthe embodiments to the precise forms disclosed. Although specificembodiments of and examples are described herein for illustrativepurposes, various equivalent modifications can be made without departingfrom the spirit and scope of the disclosure, as will be recognized bythose skilled in the relevant art. The teachings provided herein of thevarious embodiments can be applied to other systems, not necessarily theexemplary object evaluation system generally described above.

For instance, the foregoing detailed description has set forth variousembodiments of the devices and/or processes via the use of blockdiagrams, schematics, and examples. Insofar as such block diagrams,schematics, and examples contain one or more functions and/oroperations, it will be understood by those skilled in the art that eachfunction and/or operation within such block diagrams, flowcharts, orexamples can be implemented, individually and/or collectively, by a widerange of hardware, software, firmware, or virtually any combinationthereof. In one embodiment, the present subject matter may beimplemented via Application Specific Integrated Circuits (ASICs).However, those skilled in the art will recognize that the embodimentsdisclosed herein, in whole or in part, can be equivalently implementedin standard integrated circuits, as one or more computer programsrunning on one or more computers (e.g., as one or more programs runningon one or more computer systems), as one or more programs running on oneor more controllers (e.g., microcontrollers) as one or more programsrunning on one or more processors (e.g., microprocessors), as firmware,or as virtually any combination thereof.

In addition, those skilled in the art will appreciate that themechanisms taught herein are capable of being distributed as a programproduct in a variety of forms, and that an illustrative embodimentapplies equally regardless of the particular type of signal bearingmedia used to actually carry out the distribution. Examples of physicalsignal bearing media include, but are not limited to, the following:recordable type media such as floppy disks, hard disk drives, CD ROMs,digital tape, and computer memory.

The various embodiments described above can be combined to providefurther embodiments. To the extent that they are not inconsistent withthe specific teachings and definitions herein, all of the U.S. patents,U.S. patent application publications, U.S. patent applications, foreignpatents, foreign patent applications and non-patent publicationsreferred to in this specification and/or listed in the Application DataSheet that are commonly assigned to the same assignee of thisapplication are incorporated herein by reference, in their entirety.Aspects of the embodiments can be modified, if necessary, to employsystems, circuits and concepts of the various patents, applications andpublications to provide yet further embodiments.

These and other changes can be made to the embodiments in light of theabove-detailed description. In general, in the following claims, theterms used should not be construed to limit the claims to the specificembodiments disclosed in the specification and the claims, but should beconstrued to include all possible embodiments along with the full scopeof equivalents to which such claims are entitled. Accordingly, theclaims are not limited by the disclosure.

What is claimed is:
 1. A method of computationally processing images ofareas which may contain objects, the method comprising: acquiring apixelated color image of an area as a set of pixilated color image dataat least temporarily stored in at least one processor-readable medium;computationally preprocessing by at least one processor the set ofpixelated color image data for at least a portion of the area to producea set of two color contour image data that represents at least theportion of the pixilated color image, wherein a first color of the twocolor contour image data corresponds to contour pixels and a secondcolor of the two color contour image corresponds to non-contour pixels;computationally processing by the at least one processor the set of twocolor contour image data to find a location of a bottom of a stack ofobjects in the set of two color contour image data and to find alocation of a top of the stack of objects in the set of two colorcontour image data, if any objects are in the at least portion of thearea, wherein the stack of objects includes at least one object if anyobjects are in the at least portion of the area; computationallydetermining by the at least one processor a total number of objects inthe stack, if any based at least in part on a knowledge of the locationof the bottom of the stack of objects and the location of the top of thestack of objects; computationally processing by the at least oneprocessor a portion of the set of pixelated color image data based onthe knowledge of the location of the bottom of the stack of objects andthe location of the top of the stack of objects to respectively classifyeach object in the stack of objects into one of a defined set of colorclasses; and computationally determining by the at least one processor acumulative total value of the stack of objects based at least in part onthe color class to which each of the objects in the stack of objects isclassified.
 2. The method of claim 1 wherein acquiring a pixilated colorimage of an area includes capturing a color image by at least one imageacquisition device of at least a portion of a gaming table from aposition spaced at a non-zero degree and non-perpendicular angle out ofa plane of a playing surface of the gaming table.
 3. The method of claim1 wherein computationally preprocessing by at least one processor theset of pixelated color image data for at least a portion of the area toproduce a set of two color contour image data includes: converting atleast a portion of the set of the pixelated color image data to a set ofgray scale image data; and edge detection filtering the set of grayscale image data to produce a set of contour image data.
 4. The methodof claim 3 wherein computationally preprocessing by at least oneprocessor the set of pixelated color image data for at least a portionof the area to produce a set of two color contour image data furtherincludes: converting by the at least one processor at least a portion ofthe set of contour image data to a set of black and white contour imagedata.
 5. The method of claim 1 wherein computationally preprocessing byat least one processor the set of pixelated color image data for atleast a portion of the area to produce a set of two color contour imagedata further includes: eliminating extraneous pixels from the set of twocolor contour image data.
 6. The method of claim 4 wherein converting bythe at least one processor at least a portion of the set of contourimage data to a set of black and white contour image data includes:converting by the at least one processor a set of gray scale contourimage data to a set of hue, saturation, and brightness (HSB) contourimage data; and converting by the at least one processor the set of HSBcontour image data to the set of black and white contour image data. 7.The method of claim 1 wherein computationally preprocessing by at leastone processor the set of pixelated color image data for at least aportion of the area to produce a set of two color contour image dataincludes: comparing a set of pixilated color image data representing apixelated reference image of the area to the set of two color contourimage data, said reference image having no stack of objects; andremoving irregularities from the set of two color contour image dataaround a stack of objects in the two color contour image data based onthe comparing.
 8. The method of claim 7 wherein the removingirregularities from the set of two color contour image data includes:subtracting the set of pixilated color image data representing apixelated reference image of the area from the set of two color contourimage data; and adding back to the set of two color contour image dataan element of average background color of the area.
 9. The method ofclaim 1 wherein computationally preprocessing by at least one processorthe set of pixelated color image data for at least a portion of the areato produce a set of two color contour image data includes: colorbalancing a set of pixilated color image data representing a pixelatedreference image of the area to have it more closely match the set of twocolor contour image data, said reference image having no stack ofobjects; rotating the set of two color contour image data to compensatefor camera distortion for one or more stacks of objects that are nearedges of the two color contour image data; aligning the set of pixilatedcolor image data representing the pixelated reference image and the setof two color contour image data to account for movement of an imageacquisition device used to acquire the pixelated color image;subtracting the set of pixilated color image data representing apixelated reference image of the area from the set of two color contourimage data; adding back to the set of two color contour image data imagedata representing an element of average background color of the area;and either before the subtracting or after the subtracting, convertingthe set of pixilated color image data representing the pixelatedreference image of the area and the set of two color contour image datato gray scale if the converting to gray scale had been determined likelyto improve accuracy of the determining of the total number of objects inthe stack.
 10. The method of claim 1 wherein computationallypreprocessing by at least one processor the set of pixelated color imagedata for at least a portion of the area to produce a set of two colorcontour image data further includes: eliminating extraneous pixels fromthe set of two color contour image data that are outside the at leastportion of the area; and eliminating extraneous pixels that are not partof groups that at least approximately form the shape of a stack objectbased on a minimum threshold number of pixels.
 11. The method of claim10 wherein the eliminating extraneous pixels that are not part of groupscomprises: passing a template shaped as a stack object over the set oftwo color contour image data; counting a number of contour pixels withinan area defined by the template for each location of the template on theset of two color contour image data; and eliminating all pixels outsidethe area defined by the template at each location if the number ofcontour pixels within the area defined by the template contains theminimum threshold number of pixels.
 12. The method of claim 1 furthercomprising: before computationally determining by the at least oneprocessor a total number of objects in the stack, determining ahorizontal center of the stack of objects based on an overallconcentration of contour pixels in the set of two color contour imagedata; and determining an estimated height of individual objects in thestack of objects based on a mean/average height of an individual objectof the stack rationalized for a distance between an image acquisitiondevice used to acquire the pixelated color image and an expectedlocation of the stack, said distance adjusted based on the location ofthe bottom of the stack at the determined horizontal center of the stackrelative to the expected location of the stack of objects.
 13. Themethod of claim 1 wherein computationally preprocessing by at least oneprocessor the set of pixelated color image data for at least a portionof the area to produce a set of two color contour image data furtherincludes: logically segmenting the two color contour image data into aset of target pixel areas, each of the target pixel areas including aset of n by m pixels.
 14. The method of claim 13 wherein computationallypreprocessing by at least one processor the set of pixelated color imagedata for at least a portion of the area to produce a set of two colorcontour image data further includes eliminating extraneous pixels fromthe set of two color contour image data by: for each target pixel areaof the set of target pixel areas, eliminating all contour pixels in thetarget pixel area from the set of two color contour image data if atotal number of the contour pixels in the target pixel area is less thata threshold number of pixels per target pixel area; and for each targetpixel area of at least some of the set of target pixel areas,eliminating all contour pixels in the target pixel area from the set oftwo color contour image data if the target pixel area has less than anearest neighboring threshold number of nearest neighboring target pixelareas.
 15. The method of claim 13 wherein computationally processing bythe at least one processor the set of two color contour image data tofind a location of a bottom and a top of a stack of objects in the setof two color contour image data includes: applying a sample bounding boxat successive intervals to groups of target pixel areas of the set oftarget pixel areas to find a location of the bottom of the stack ofobjects, each of the groups of target pixel areas including at leastsome target pixel areas with contour pixels of the set of two colorcontour image data that are below a determined upper location in the setof two color contour image data; determining a bottom quadrant to be thesample bounding box that includes the group of target pixel areas withthe highest number of target pixel areas having contour pixels;determining whether a width of the target pixel areas included in thebottom quadrant fits a face ellipse of an object; determining whether awidth of the target pixel areas included bottom quadrant fits a faceellipse of an object; determining that there is no stack present whenthe width does not fit the face ellipse; and determining a location ofthe bottom of the bottom quadrant when the width fits the face ellipse.16. The method of claim 15 wherein computationally processing by the atleast one processor the set of two color contour image data to find alocation of a bottom and a top of a stack of objects in the set of twocolor contour image data includes: determining a location of a bottom ofthe bottom quadrant by traversing downwardly starting at a locationoffset from the determined upper location until a last one or moretarget pixel areas found within a left and right boundary of the bottomquadrant.
 17. The method of claim 13 wherein computationally processingby the at least one processor the set of two color contour image data tofind a location of a bottom and a top of a stack of objects in the setof two color contour image data includes: successively applying a samplebounding box to groups of target pixel areas of the set of target pixelareas at successive intervals from a determined bottom of the stack ofobjects until a top streak of target pixel areas contained within thesample bounding box is empty.
 18. The method of claim 17 whereincomputationally processing by the at least one processor the set of twocolor contour image data to find a location of a bottom and a top of astack of objects in the set of two color contour image data includes:determining a top bounding box based on the values of the samplebounding box with the sample bounding box positioned such that the topstreak of target quadrants contained within the sample bounding box isempty; successively applying the sample bounding box at successiveintervals to find a sample quadrant that fits an elliptical pattern;determining a list of coordinates of the target quadrants that intersecta top arc and a bottom arc of an imaginary ellipse the size of thesample bounding box; determining a total number of target quads thatoverlap the target quadrants of the imaginary ellipse from the samplebounding box; and determining the sample bounding box position with themost number of overlaps as the top quadrants.
 19. The method of claim 18wherein computationally determining by the at least one processor atotal number of objects in the stack includes determining a differencein the bottom of the top quadrant and a bottom of a bottom quadrant anda mean/average height of an individual object rationalized for adistance between the image acquisition device and the stack of objects.20. The method of claim 1 wherein computationally processing by the atleast one processor a portion of the set of pixelated color image databased on a knowledge of a location of a bottom of the stack of objectsand a location of a top of the stack of objects to respectively classifyeach object in the stack of objects into one of a defined set of colorclasses includes: for each of the objects, classifying a number ofpixels in a color pixel streak to respective hue, saturation, andbrightness (HSB) values that represent the hue, saturation andbrightness of the number of pixels, the color pixel streak composed of asubset of the pixels between a top arc and a bottom arc of an ellipse ofthe object.
 21. The method of claim 20 wherein computationallyprocessing by the at least one processor a portion of the set ofpixelated color image data based on the knowledge of the location of thebottom of the stack of objects and the location of the top of the stackof objects to respectively classify each object in the stack of objectsinto one of a defined set of color classes further includes: for each ofthe objects, classifying the number of pixels in the color pixel streakto respective fundamental colors based on defined relationships betweenHSB values and a defined set of fundamental colors.
 22. The method ofclaim 21 wherein computationally processing by the at least oneprocessor a portion of the set of pixelated color image data based onthe knowledge of the location of the bottom of the stack of objects andthe location of the top of the stack of objects to respectively classifyeach object in the stack of objects into one of a defined set of colorclasses further includes: for each of the objects, determining whetherthe classified fundamental colors of the color pixel streak is in a setof color drift colors.
 23. The method of claim 22 whereincomputationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes furtherincludes: for each of the objects, determining whether the classifiedfundamental colors of the color pixel streak are a defined primaryfundamental color and a number of defined secondary fundamental colors.24. The method of claim 23 wherein computationally processing by the atleast one processor a portion of the set of pixelated color image databased on the knowledge of the location of the bottom of the stack ofobjects and the location of the top of the stack of objects torespectively classify each object in the stack of objects into one of adefined set of color classes further includes: for each of the objects,if the classified fundamental colors of the color pixel streak are notin the set of color drift colors and the classified fundamental colorsof the color pixel streak are not one of the defined primary fundamentalcolor or the defined secondary fundamental colors, determining afundamental color based on a most common color shared by the pixels ofthe color pixel streak.
 25. The method of claim 23 whereincomputationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes furtherincludes: for each of the objects, determining a value of the objectbased on a determined fundamental color and a defined set of fundamentalcolor to object value relationships.
 26. The method of claims 24 whereincomputationally processing by the at least one processor a portion ofthe set of pixelated color image data based on the knowledge of thelocation of the bottom of the stack of objects and the location of thetop of the stack of objects to respectively classify each object in thestack of objects into one of a defined set of color classes furtherincludes: summing the determined values of each of the objects.
 27. Themethod of claim 1 further comprising: reclassifying an object into adifferent color class based on a determination that a color of theobject is obscured by a shadow in the pixelated color image.
 28. Themethod of claim 27 wherein the reclassifying an object into a differentcolor class includes: determining whether pixels of an object previouslyclassified as black surpass a threshold regarding (HSB) values of thepixels; and reclassifying the object into a different color class if thethreshold is surpassed; and the method further includes: re-determininga value of the object based on the reclassification.
 29. A system,comprising: at least one image capturing device configured to acquire apixelated color image of an area as a set of pixelated color image data;and at least one object evaluation computing system having at least oneprocessor and at least one storage medium that at least temporarilystores a the set of pixelated color image data, the object evaluationcomputing system configured to: preprocess using the at least oneprocessor the set of pixelated color image data for at least a portionof the area to produce a set of two color contour image data thatrepresents at least the portion of the pixilated color image; processusing the at least one processor the set of two color contour image datato find a location of a bottom of a stack of objects in the set of twocolor contour image data and to find a location of a top of the stack ofobjects in the set of two color contour image data, if any objects arein the at least portion of the area, wherein the stack of objectsincludes at least one object if any objects are in the at least portionof the area; determine using the at least one processor a total numberof objects in the stack, if any based at least in part on a knowledge ofthe location of the bottom of the stack of objects and the location ofthe top of the stack of objects; process using the at least oneprocessor a portion of the set of pixelated color image data based onthe knowledge of the location of the bottom of the stack of objects andthe location of the top of the stack of objects to respectively classifyeach object in the stack of objects into one of a defined set of colorclasses; and determine using the at least one processor a cumulativetotal value of the stack of objects based at least in part on the colorclass to which each of the objects in the stack of objects isclassified.
 30. The system of claim 29, wherein the at least one objectevaluation computing system is further configured to convert at least aportion of the set of the pixelated color image data to a set of grayscale image data and perform edge detection filtering of the set of grayscale image data to produce a set of contour image data.
 31. The systemof claim 29, wherein the at least one object evaluation computing systemis further configured to logically segment the two color contour imagedata into a set of target pixel areas, each of the target pixel areasincluding a set of n by m pixels.
 32. The system of claim 29 wherein theat least one object evaluation computing system is configured topreprocess using the at least one processor the set of pixelated colorimage data by: comparing a set of pixilated color image datarepresenting a pixelated reference image of the area to the set of twocolor contour image data, said reference image having no stack ofobjects; and removing irregularities from the set of two color contourimage data around a stack of objects in the two color contour image databased on the comparing.
 33. The system of claim 32 wherein the removingirregularities comprises: subtracting the set of pixilated color imagedata representing a pixelated reference image of the area from the setof two color contour image data; and adding back to the set of two colorcontour image data an element of average background color of the area.34. The system of claim 29 wherein the at least one object evaluationcomputing system is configured to preprocess using the at least oneprocessor the set of pixelated color image data by: color balancing aset of pixilated color image data representing a pixelated referenceimage of the area to have it more closely match the set of two colorcontour image data, said reference image having no stack of objects;rotating the set of two color contour image data to compensate forcamera distortion for one or more stacks of objects that are near edgesof the two color contour image data; aligning the set of pixilated colorimage data representing the pixelated reference image and the set of twocolor contour image data to account for movement of an image acquisitiondevice used to acquire the pixelated color image; subtracting the set ofpixilated color image data representing a pixelated reference image ofthe area from the set of two color contour image data; adding back tothe set of two color contour image data image data representing anelement of average background color of the area; and either before thesubtracting or after the subtracting, converting the set of pixilatedcolor image data representing the pixelated reference image of the areaand the set of two color contour image data to gray scale if theconverting to gray scale had been determined likely to improve accuracyof the determining of the total number of objects in the stack.
 35. Thesystem of claim 29 wherein the at least one object evaluation computingsystem is configured to preprocess using the at least one processor theset of pixelated color image data by: eliminating extraneous pixels fromthe set of two color contour image data that are outside the at leastportion of the area; and eliminating extraneous pixels that are not partof groups that at least approximately form the shape of a stack objectbased on a minimum threshold number of pixels.
 36. The system of claim35 the eliminating extraneous pixels that are not part of groupscomprises: passing a template shaped as a stack object over the set oftwo color contour image data; counting a number of contour pixels withinan area defined by the template for each location of the template on theset of two color contour image data; and eliminating all pixels outsidethe area defined by the template at each location if the number ofcontour pixels within the area defined by the template contains theminimum threshold number of pixels.
 37. The system of claim 29 whereinthe at least one object evaluation computing system is furtherconfigured to: before computationally determining using the at least oneprocessor a total number of objects in the stack, determine a horizontalcenter of the stack of objects based on an overall concentration ofcontour pixels in the set of two color contour image data; and determinean estimated height of individual objects in the stack of objects basedon a mean/average height of an individual object of the stackrationalized for a distance between an image acquisition device used toacquire the pixelated color image and an expected location of the stack,said distance adjusted based on the location of the bottom of the stackat the determined horizontal center of the stack relative to theexpected location of the stack of objects.
 38. The system of claim 37,wherein the at least one object evaluation computing system is furtherconfigured to eliminate extraneous pixels from the set of two colorcontour image data by: for each target pixel area of the set of targetpixel areas, eliminating all contour pixels in the target pixel areafrom the set of two color contour image data if a total number of thecontour pixels in the target pixel area is less that a threshold numberof pixels per target pixel area; and for each target pixel area of atleast some of the set of target pixel areas, eliminating all contourpixels in the target quadrant from the set of two color contour imagedata if the target pixel area has less than a nearest neighboringthreshold number of nearest neighboring target pixel areas.
 39. Thesystem of claim 37, wherein the at least one object evaluation computingsystem is further configured to: apply a sample bounding box atsuccessive intervals to groups of target pixel areas of the set oftarget pixel areas to find a location of the bottom the stack ofobjects, each of the groups of target pixel areas including at leastsome target pixel areas with contour pixels of the set of two colorcontour image data that are below a determined upper location in the setof two color contour image data; determine a bottom quadrant to be thesample bounding box that includes the group of target pixel areas withthe highest number of target pixel areas having contour pixels;determine whether a width of the target pixel areas included in thebottom quadrant fits a face ellipse of an object; determine whether awidth of the target pixel areas included bottom quadrant fits a faceellipse of an object; determine that there is no stack present when thewidth does not fit the face ellipse; and determine a location of thebottom of the bottom quadrant when the width fits the face ellipse. 40.The system of claim 37, wherein the at least one object evaluationcomputing system is further configured to: successively apply a samplebounding box to groups of target pixel areas of the set of target pixelareas at successive intervals from a determined bottom of the stack ofobjects until a top streak of target pixel areas contained within thesample bounding box is empty.
 41. The system of claim 40, wherein the atleast one object evaluation computing system is further configured to:determine a top bounding box based on the values of the sample boundingbox with the sample bounding box positioned such that the top streak oftarget quadrants contained within the sample bounding box is empty;successively apply the sample bounding box at successive intervals tofind a sample quadrant that fits an elliptical pattern; determine a listof coordinates of the target quadrants that intersect a top arc and abottom arc of an imaginary ellipse the size of the sample bounding box;determine a total number of target quads that overlap the targetquadrants of the imaginary ellipse from the sample bounding box; anddetermine the sample bounding box position with the most number ofoverlaps as the top quadrants.
 42. The system of claim 41, wherein theat least one object evaluation computing system is further configuredto: determine a total number of objects in the stack by determining adifference in the bottom of the top quadrant and a bottom of a bottomquadrant and a mean/average height of an individual object rationalizedfor a distance between the image acquisition device and the stack ofobjects.
 43. The system of claim 29, wherein the at least one objectevaluation computing system is further configured to: classify for eachof the objects a number of pixels in a color pixel streak to respectivehue, saturation, and brightness (HSB) values that represent the hue,saturation and brightness of the number of pixels, the color pixelstreak composed of a subset of the pixels between a top arc and a bottomarc of an ellipse of the object.
 44. The system of claim 43, wherein theat least one object evaluation computing system is further configuredto: classify for each of the objects the number of pixels in the colorpixel streak to respective fundamental colors based on definedrelationships between HSB values and a defined set of fundamentalcolors.
 45. The system of claim 44, wherein the at least one objectevaluation computing system is further configured to: determine for eachof the objects whether the classified fundamental colors of the colorpixel streak is in a set of color drift colors.
 46. The system of claim43, wherein the at least one object evaluation computing system isfurther configured to: determine for each of the objects a value of theobject based on a determined fundamental color and a defined set offundamental color to object value relationships.
 47. The system of claim29 wherein the at least one object evaluation computing system isfurther configured to: reclassify an object into a different color classbased on a determination that a color of the object is obscured by ashadow in the pixelated color image.
 48. The system of claim 47 whereinthe at least one object evaluation computing system is configured toreclassify an object by: determining whether pixels of an objectpreviously classified as black surpass a threshold regarding (HSB)values of the pixels; and reclassifying the object into a differentcolor class if the threshold is surpassed; and the at least one objectevaluation computing system may be further configured to: re-determine avalue of the object based on the reclassification.